Jul 09 09:52:16.190860 ip-10-0-134-220 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 09 09:52:16.190869 ip-10-0-134-220 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 09 09:52:16.190876 ip-10-0-134-220 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 09 09:52:16.191091 ip-10-0-134-220 systemd[1]: Failed to start Kubernetes Kubelet. Jul 09 09:52:25.703235 ip-10-0-134-220 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 09 09:52:25.703243 ip-10-0-134-220 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c406753278654e80a1ede306c2089f4b -- Jul 09 09:54:12.639918 ip-10-0-134-220 systemd[1]: Starting Kubernetes Kubelet... Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.738702 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743806 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743819 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743826 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743831 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743834 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743838 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743841 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 09:54:13.792010 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743845 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743848 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743851 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743854 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743857 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743861 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743864 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743866 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743869 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743872 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743875 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743878 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743881 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743883 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743886 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743889 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743892 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743899 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743902 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743905 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 09:54:13.793134 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743908 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743911 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743913 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743916 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743919 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743922 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743925 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743928 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743932 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743936 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743939 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743942 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743944 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743947 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743950 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743954 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743958 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743961 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743964 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 09:54:13.794712 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743966 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743969 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743972 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743975 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743978 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743980 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743983 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743985 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743988 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743991 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743994 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.743998 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744000 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744003 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744006 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744010 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744013 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744016 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744019 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744022 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 09:54:13.795287 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744025 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744028 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744030 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744033 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744036 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744039 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744042 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744046 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744050 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744054 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744056 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744060 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744062 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744065 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744068 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744071 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744074 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744077 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744080 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744083 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 09:54:13.795933 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744086 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744089 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744093 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744096 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744098 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744101 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744104 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744107 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744612 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744617 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744622 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744625 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.744629 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745407 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745416 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745422 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745425 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745428 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745431 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 09:54:13.796546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745434 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745437 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745440 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745443 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745446 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745448 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745451 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745454 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745456 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745459 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745462 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745465 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745467 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745470 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745473 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745476 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745479 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745482 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745484 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745487 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 09:54:13.797116 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745490 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745494 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745497 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745499 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745502 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745505 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745508 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745511 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745514 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745516 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745519 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745522 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745525 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745528 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745531 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745533 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745536 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745539 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745542 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745545 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 09:54:13.797755 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745547 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745550 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745553 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745556 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745558 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745561 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745564 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745566 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745570 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745573 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745576 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745580 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745583 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745585 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745588 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745591 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745594 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745597 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745601 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745603 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 09:54:13.798361 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745606 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745609 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745611 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745615 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745618 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745622 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745624 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745627 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745630 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745636 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745638 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745641 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745644 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745647 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745651 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745654 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745657 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745659 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745663 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 09:54:13.799044 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745668 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745671 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745675 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745678 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745681 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745704 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745708 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745711 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745714 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745717 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745720 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745723 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745726 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.745729 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746153 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746159 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746162 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746166 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746176 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746179 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 09:54:13.799618 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746182 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746185 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746190 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746193 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746213 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746217 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746221 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746224 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746227 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746232 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746236 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746239 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746242 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746245 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746248 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746251 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746254 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746257 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746259 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 09:54:13.800193 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746262 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746265 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746268 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746273 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746276 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746279 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746282 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746285 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746288 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746290 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746293 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746296 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746302 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746305 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746308 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746312 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746315 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746317 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746321 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 09:54:13.800777 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746325 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746328 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746332 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746334 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746337 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746340 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746343 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746346 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746349 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746351 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746355 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746359 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746362 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746365 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746368 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746370 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746375 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746378 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746381 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 09:54:13.801396 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746383 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746386 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746389 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746391 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746394 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746397 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746399 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746402 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746405 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746409 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746412 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746415 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746417 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746420 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746423 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746426 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746428 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746431 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746435 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746438 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 09:54:13.801993 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746441 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746444 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746446 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746449 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746452 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746455 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746457 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746460 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746463 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746467 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746469 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746472 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746475 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746477 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746480 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.746483 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746550 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746557 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746564 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746569 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746574 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 09:54:13.802619 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746579 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746584 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746589 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746592 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746596 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746599 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746603 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746606 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746609 2582 flags.go:64] FLAG: --cgroup-root="" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746612 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746615 2582 flags.go:64] FLAG: --client-ca-file="" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746618 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746622 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746626 2582 flags.go:64] FLAG: --cluster-domain="" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746629 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746632 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746636 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746639 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746643 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746646 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746651 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746654 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746657 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746661 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 09:54:13.803253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746664 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746667 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746670 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746675 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746678 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746681 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746698 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746702 2582 flags.go:64] FLAG: --enable-server="true" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746710 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746716 2582 flags.go:64] FLAG: --event-burst="100" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746719 2582 flags.go:64] FLAG: --event-qps="50" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746723 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746726 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746729 2582 flags.go:64] FLAG: --eviction-hard="" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746733 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746736 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746739 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746742 2582 flags.go:64] FLAG: --eviction-soft="" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746745 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746749 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746752 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746755 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746758 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746762 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746765 2582 flags.go:64] FLAG: --feature-gates="" Jul 09 09:54:13.803985 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746768 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746772 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746776 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746781 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746784 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746787 2582 flags.go:64] FLAG: --help="false" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746790 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-134-220.ec2.internal" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746794 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746797 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746800 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746804 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746807 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746810 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746814 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746816 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746821 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746825 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746828 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746832 2582 flags.go:64] FLAG: --kube-reserved="" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746835 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746838 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746841 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746844 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746847 2582 flags.go:64] FLAG: --lock-file="" Jul 09 09:54:13.913199 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746850 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 09:54:13.822023 ip-10-0-134-220 systemd[1]: Started Kubernetes Kubelet. Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746853 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746856 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746862 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746865 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746868 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746871 2582 flags.go:64] FLAG: --logging-format="text" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746874 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746877 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746881 2582 flags.go:64] FLAG: --manifest-url="" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746885 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746892 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746895 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746899 2582 flags.go:64] FLAG: --max-pods="110" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746902 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746906 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746909 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746912 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746916 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746919 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746922 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746929 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746932 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746936 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 09:54:13.914847 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746940 2582 flags.go:64] FLAG: --pod-cidr="" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746944 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746950 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746953 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746956 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746959 2582 flags.go:64] FLAG: --port="10250" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746962 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746966 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ffeff5bd1b17d15c" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746969 2582 flags.go:64] FLAG: --qos-reserved="" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746972 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746975 2582 flags.go:64] FLAG: --register-node="true" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746978 2582 flags.go:64] FLAG: --register-with-taints="" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746982 2582 flags.go:64] FLAG: --registry-burst="10" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746985 2582 flags.go:64] FLAG: --registry-qps="5" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746988 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746992 2582 flags.go:64] FLAG: --reserved-memory="" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746995 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.746999 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747002 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747007 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747010 2582 flags.go:64] FLAG: --runonce="false" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747013 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747016 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747020 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747023 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 09:54:13.915826 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747026 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747029 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747032 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747036 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747039 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747042 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747045 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747049 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747052 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747056 2582 flags.go:64] FLAG: --system-cgroups="" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747059 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747064 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747068 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747071 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747075 2582 flags.go:64] FLAG: --tls-min-version="" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747078 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747081 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747084 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747087 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747090 2582 flags.go:64] FLAG: --v="2" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747094 2582 flags.go:64] FLAG: --version="false" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747098 2582 flags.go:64] FLAG: --vmodule="" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747102 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747105 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747236 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 09:54:13.916867 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747241 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747246 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747249 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747252 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747255 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747257 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747260 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747263 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747266 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747269 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747272 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747274 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747277 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747280 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747285 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747287 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747292 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747296 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747299 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 09:54:13.918211 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747302 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747305 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747308 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747311 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747314 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747317 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747320 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747322 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747325 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747328 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747331 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747334 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747337 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747339 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747344 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747347 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747350 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747353 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747356 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747358 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 09:54:13.919157 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747361 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747364 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747367 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747370 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747374 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747377 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747381 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747385 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747388 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747390 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747393 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747397 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747399 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747403 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747407 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747410 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747413 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747416 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747419 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 09:54:13.919968 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747422 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747425 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747428 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747430 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747433 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747458 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747461 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747465 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747468 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747470 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747473 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747476 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747479 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747482 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747485 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747487 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747490 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747493 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747496 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747498 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 09:54:13.920655 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747502 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747505 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747508 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747510 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747513 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747517 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747520 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747523 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747525 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747528 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747531 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747534 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747537 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747539 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747542 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747545 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747548 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747550 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747553 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 09:54:13.921398 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.747557 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.747566 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.755529 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.755544 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.756232 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.756240 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756300 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756305 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756309 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756313 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756315 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756318 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756321 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756324 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756327 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 09:54:13.922110 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756330 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756333 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756336 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756340 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756343 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756346 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756348 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756351 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756356 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756360 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756363 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756367 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756372 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756375 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756378 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756381 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756384 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756390 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756394 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 09:54:13.922747 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756396 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756399 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756402 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756405 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756408 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756411 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756414 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756417 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756420 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756423 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756425 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756428 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756431 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756434 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756437 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756440 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756442 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756446 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756449 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756453 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 09:54:13.923461 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756456 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756458 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756461 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756464 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756467 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756470 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756472 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756475 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756478 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756496 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756502 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756505 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756509 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756512 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756515 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756518 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756522 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756525 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756528 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756530 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 09:54:13.924189 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756533 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756536 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756539 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756542 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756545 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756547 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756550 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756553 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756556 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756558 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756562 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756566 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756570 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756572 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756575 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756578 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756580 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756585 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756589 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 09:54:13.924948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756592 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756594 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756597 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756603 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756606 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756610 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756613 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756617 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756621 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756624 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756627 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756630 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.756636 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756787 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756792 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 09:54:13.925546 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756796 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756799 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756802 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756804 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756808 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756810 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756813 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756817 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756821 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756826 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756829 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756832 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756835 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756838 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756840 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756843 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756846 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756849 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756851 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 09:54:13.926111 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756854 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756857 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756861 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756864 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756866 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756869 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756872 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756875 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756878 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756880 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756883 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756886 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756889 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756891 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756894 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756897 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756899 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756902 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756905 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756908 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 09:54:13.927032 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756911 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756913 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756918 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756921 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756924 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756927 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756947 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756951 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756954 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756957 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756960 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756963 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756966 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756969 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756973 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756976 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756979 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756981 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756985 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756989 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 09:54:13.928187 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756992 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756995 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.756998 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757001 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757004 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757007 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757010 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757013 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757015 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757020 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757023 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757026 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757029 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757032 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757036 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757040 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757044 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757047 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757050 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 09:54:13.928868 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757053 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757055 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757058 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757061 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757063 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757066 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757069 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757072 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757075 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757078 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757081 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757084 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757086 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757089 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757092 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757095 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757097 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757100 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 09:54:13.929985 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:13.757103 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 09 09:54:13.930660 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.757107 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 09:54:13.930660 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.757213 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 09:54:13.930660 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.759299 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 09:54:13.930660 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.760473 2582 server.go:1019] "Starting client certificate rotation" Jul 09 09:54:13.930660 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.760581 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 09:54:13.930660 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.760629 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 09:54:13.930660 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.762605 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 09:54:13.930660 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.773339 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 09:54:13.930660 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.781866 2582 log.go:25] "Validated CRI v1 runtime API" Jul 09 09:54:13.930660 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.783948 2582 log.go:25] "Validated CRI v1 image API" Jul 09 09:54:13.930660 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.785425 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 09:54:13.930660 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.787584 2582 fs.go:135] Filesystem UUIDs: map[3ea7aaa3-f783-469f-b8eb-d0bf6f1b52c1:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c1bf5ae7-278a-4590-ad65-b1a4807e1843:/dev/nvme0n1p3] Jul 09 09:54:13.931114 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.787601 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 09:54:13.931114 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.789220 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.794844 2582 manager.go:216] Machine: {Timestamp:2026-07-09 09:54:13.794321075 +0000 UTC m=+1.053270930 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100952 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20a893c51720168a14082c2de4d825 SystemUUID:ec20a893-c517-2016-8a14-082c2de4d825 BootID:c4067532-7865-4e80-a1ed-e306c2089f4b Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1b:ec:b1:20:93 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1b:ec:b1:20:93 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fe:7b:84:88:08:b9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.794947 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.795063 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.797891 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.797921 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-220.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.798649 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.798660 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.798668 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.799284 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.800642 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.804754 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.804887 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.804923 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.804938 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.804952 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.804963 2582 kubelet.go:398] "Adding apiserver pod source" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.804974 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 09:54:13.931565 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.805350 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.805373 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.813054 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.814939 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.814969 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.815217 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2wppb" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816159 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816179 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816186 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816194 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816204 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816210 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816219 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816225 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816236 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816242 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816257 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816262 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816918 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.816925 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:13.817963 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:13.819343 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-220.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.820953 2582 server.go:1268] "Started kubelet" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.821025 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.821107 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.821193 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.821942 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2wppb" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.832134 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:13.833510 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.834273 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.838533 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-220.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.838934 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.838958 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:13.839039 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:13.838323 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-220.ec2.internal.18c0966e829795dd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-220.ec2.internal,UID:ip-10-0-134-220.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-220.ec2.internal,},FirstTimestamp:2026-07-09 09:54:13.820257757 +0000 UTC m=+1.079207609,LastTimestamp:2026-07-09 09:54:13.820257757 +0000 UTC m=+1.079207609,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-220.ec2.internal,}" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.839548 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.839580 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.839598 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.839731 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.839741 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 09 09:54:13.932445 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.839855 2582 factory.go:55] Registering systemd factory Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.839871 2582 factory.go:223] Registration of the systemd container factory successfully Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.840077 2582 factory.go:153] Registering CRI-O factory Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.840088 2582 factory.go:223] Registration of the crio container factory successfully Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.840141 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.840166 2582 factory.go:103] Registering Raw factory Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.840182 2582 manager.go:1196] Started watching for new ooms in manager Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.840611 2582 manager.go:349] Starting recovery of all containers Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.843442 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.851038 2582 manager.go:354] Recovery completed Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:13.852189 2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-220.ec2.internal\" not found" node="ip-10-0-134-220.ec2.internal" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.855619 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.859052 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.859088 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.859099 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.860117 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.860128 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.860133 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.860145 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.862440 2582 policy_none.go:49] "None policy: Start" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.862453 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.862464 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.864557 2582 policy_none.go:47] "Start" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.865744 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.865764 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.865781 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.916401 2582 manager.go:337] "Starting Device Plugin manager" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:13.916435 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.916457 2582 server.go:87] "Starting device plugin registration server" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.916763 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.916777 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.916879 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.916976 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:13.916986 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:13.917672 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 09:54:13.934097 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:13.917738 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-220.ec2.internal\" not found" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.020124 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.021262 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.021294 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.021310 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.021343 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.031067 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.031091 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-220.ec2.internal\": node \"ip-10-0-134-220.ec2.internal\" not found" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.032873 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.034185 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.034234 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.034260 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.034301 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.036650 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:14.075800 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.046723 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.141191 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal"] Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.141270 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.146599 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.146647 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.146658 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.146755 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.149234 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.149356 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.149396 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.155796 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.155816 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.155826 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.155839 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.155841 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.155852 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.158270 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.158298 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.159030 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientMemory" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.159055 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.159067 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeHasSufficientPID" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.179270 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-220.ec2.internal\" not found" node="ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.205983 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.182994 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-220.ec2.internal\" not found" node="ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.321510 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.246703 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4780b8f2df1828ad4bb2a95b7e56dc50-config\") pod \"kube-apiserver-proxy-ip-10-0-134-220.ec2.internal\" (UID: \"4780b8f2df1828ad4bb2a95b7e56dc50\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.321510 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.246740 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/18a90544793a6b2ed8c167388511c8cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal\" (UID: \"18a90544793a6b2ed8c167388511c8cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.321510 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.246760 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18a90544793a6b2ed8c167388511c8cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal\" (UID: \"18a90544793a6b2ed8c167388511c8cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.321510 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.246808 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found" Jul 09 09:54:14.399340 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.347431 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found" Jul 09 09:54:14.399340 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.347474 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/18a90544793a6b2ed8c167388511c8cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal\" (UID: \"18a90544793a6b2ed8c167388511c8cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.399340 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.347502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18a90544793a6b2ed8c167388511c8cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal\" (UID: \"18a90544793a6b2ed8c167388511c8cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.399340 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.347521 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4780b8f2df1828ad4bb2a95b7e56dc50-config\") pod \"kube-apiserver-proxy-ip-10-0-134-220.ec2.internal\" (UID: \"4780b8f2df1828ad4bb2a95b7e56dc50\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.399340 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.347570 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4780b8f2df1828ad4bb2a95b7e56dc50-config\") pod \"kube-apiserver-proxy-ip-10-0-134-220.ec2.internal\" (UID: \"4780b8f2df1828ad4bb2a95b7e56dc50\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.399340 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.347574 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/18a90544793a6b2ed8c167388511c8cc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal\" (UID: \"18a90544793a6b2ed8c167388511c8cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.399340 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.347603 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18a90544793a6b2ed8c167388511c8cc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal\" (UID: \"18a90544793a6b2ed8c167388511c8cc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.474487 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.447981 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found" Jul 09 09:54:14.550607 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.482193 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.550607 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.486441 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.550607 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.548454 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found" Jul 09 09:54:14.648722 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.648685 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-220.ec2.internal\" not found" Jul 09 09:54:14.676097 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.676076 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:14.740457 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.740386 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.755142 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.755114 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 09:54:14.756684 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.756671 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.760680 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.760664 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 09:54:14.760814 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.760796 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 09:54:14.760878 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.760838 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 09:54:14.760878 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.760838 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 09:54:14.760946 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.760881 2582 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://ab8225b3aa6dd4799b93e6657fadfa59-7c24328bfda0290d.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.134.220:44668->18.214.161.245:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" Jul 09 09:54:14.805566 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.805541 2582 apiserver.go:52] "Watching apiserver" Jul 09 09:54:14.828518 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.828487 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 09:54:14.829745 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.829722 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-f987x","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g","openshift-cluster-node-tuning-operator/tuned-j68cp","openshift-image-registry/node-ca-grjk4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal","openshift-multus/multus-bjdg8","openshift-multus/network-metrics-daemon-7fx9w","openshift-ovn-kubernetes/ovnkube-node-psrgx","kube-system/konnectivity-agent-v97nr","openshift-dns/node-resolver-chwp7","openshift-multus/multus-additional-cni-plugins-nktqq","openshift-network-diagnostics/network-check-target-fg9gs","openshift-network-operator/iptables-alerter-jp9bl"] Jul 09 09:54:14.832495 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.832477 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:14.832585 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.832564 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:14.834792 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.834773 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.837134 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.837114 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.837475 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.837449 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 09:49:14 +0000 UTC" deadline="2028-04-05 08:03:07.03561319 +0000 UTC" Jul 09 09:54:14.837475 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.837475 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15262h8m52.198140938s" Jul 09 09:54:14.837612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.837552 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 09:54:14.837798 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.837780 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 09:54:14.837862 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.837820 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n4brl\"" Jul 09 09:54:14.838556 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.838541 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 09:54:14.839316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.839291 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 09:54:14.839564 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.839549 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-grjk4" Jul 09 09:54:14.840284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.840113 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 09:54:14.840284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.840134 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 09:54:14.840720 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.840570 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-csfhq\"" Jul 09 09:54:14.842823 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.842803 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.842941 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.842893 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5l75b\"" Jul 09 09:54:14.843005 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.842937 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 09:54:14.843592 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.843574 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 09:54:14.844798 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.844785 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 09:54:14.845461 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.845446 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v9q5z\"" Jul 09 09:54:14.845637 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.845624 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 09:54:14.845708 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.845674 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:14.845748 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.845731 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:14.846151 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.846132 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 09:54:14.846309 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.846289 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 09:54:14.846524 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.846507 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 09:54:14.848196 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.848181 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.849803 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.849787 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csjdn\" (UniqueName: \"kubernetes.io/projected/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-kube-api-access-csjdn\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:14.849883 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.849811 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-device-dir\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.849883 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.849829 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5zv7\" (UniqueName: \"kubernetes.io/projected/b5ab5674-a77b-456f-a530-e47b0fb0e33c-kube-api-access-q5zv7\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.849883 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.849846 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-sysconfig\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.849883 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.849876 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-multus-cni-dir\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.850025 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.849902 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-os-release\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.850025 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.849920 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65dd7a41-8045-4903-bfbd-273242802bca-multus-daemon-config\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.850025 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.849938 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-run\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850025 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.849953 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-var-lib-kubelet\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850025 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.849969 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p744j\" (UniqueName: \"kubernetes.io/projected/8017af28-6540-46c3-9bc3-8b4ab3b83303-kube-api-access-p744j\") pod \"node-ca-grjk4\" (UID: \"8017af28-6540-46c3-9bc3-8b4ab3b83303\") " pod="openshift-image-registry/node-ca-grjk4" Jul 09 09:54:14.850025 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.849986 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-var-lib-kubelet\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.850025 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850022 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-hostroot\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.850230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850041 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-run-multus-certs\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.850230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850056 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8017af28-6540-46c3-9bc3-8b4ab3b83303-host\") pod \"node-ca-grjk4\" (UID: \"8017af28-6540-46c3-9bc3-8b4ab3b83303\") " pod="openshift-image-registry/node-ca-grjk4" Jul 09 09:54:14.850230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850071 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-etc-kubernetes\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.850230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850086 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-kubernetes\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850101 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-host\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850115 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/439a6e71-546c-4642-a628-15476b3deeeb-etc-tuned\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850128 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-cnibin\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.850230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850157 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-var-lib-cni-multus\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.850230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850182 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-sysctl-conf\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850199 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-systemd\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850218 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-lib-modules\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850232 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-system-cni-dir\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850247 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-multus-conf-dir\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850263 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-sys-fs\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850278 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-modprobe-d\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850293 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-run-k8s-cni-cncf-io\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850327 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850350 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850365 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-socket-dir\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850392 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-registration-dir\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850417 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-etc-selinux\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850440 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-sys\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2ndk\" (UniqueName: \"kubernetes.io/projected/439a6e71-546c-4642-a628-15476b3deeeb-kube-api-access-j2ndk\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850498 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-sysctl-d\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850517 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/439a6e71-546c-4642-a628-15476b3deeeb-tmp\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850564 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8017af28-6540-46c3-9bc3-8b4ab3b83303-serviceca\") pod \"node-ca-grjk4\" (UID: \"8017af28-6540-46c3-9bc3-8b4ab3b83303\") " pod="openshift-image-registry/node-ca-grjk4" Jul 09 09:54:14.850586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850592 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v97nr" Jul 09 09:54:14.851301 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850606 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65dd7a41-8045-4903-bfbd-273242802bca-cni-binary-copy\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.851301 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850655 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-multus-socket-dir-parent\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.851301 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850682 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-run-netns\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.851301 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850708 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-var-lib-cni-bin\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.851301 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850732 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-267hw\" (UniqueName: \"kubernetes.io/projected/65dd7a41-8045-4903-bfbd-273242802bca-kube-api-access-267hw\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.851301 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850757 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 09:54:14.851301 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.850821 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 09:54:14.851937 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.851920 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 09:54:14.852024 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.851947 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8rnsd\"" Jul 09 09:54:14.852253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.852234 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 09:54:14.852317 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.852252 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 09:54:14.852317 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.852239 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 09:54:14.853016 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.852999 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-chwp7" Jul 09 09:54:14.853377 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.853362 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 09:54:14.853529 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.853507 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 09:54:14.853529 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.853525 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5wlx7\"" Jul 09 09:54:14.855599 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.855575 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:14.856025 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.855998 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 09:54:14.856629 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.856597 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-z2js2\"" Jul 09 09:54:14.856860 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.856843 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 09:54:14.858168 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.858149 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:14.858272 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.858213 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:14.858272 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.858265 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 09:54:14.858394 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.858270 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 09:54:14.858519 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.858502 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-w4h5c\"" Jul 09 09:54:14.858951 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.858933 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 09:54:14.860841 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.860825 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jp9bl" Jul 09 09:54:14.864763 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.864743 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 09:54:14.864763 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.864754 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 09:54:14.864868 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.864797 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-h886w\"" Jul 09 09:54:14.864868 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.864755 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 09:54:14.884042 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.884021 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4fwfm" Jul 09 09:54:14.894618 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.894600 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4fwfm" Jul 09 09:54:14.941413 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.941393 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 09:54:14.949718 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.949681 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:14.951783 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.951763 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-kubernetes\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.951857 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.951801 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-var-lib-cni-multus\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.951857 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.951831 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-log-socket\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.952012 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.951858 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af327940-c861-4625-a6bb-1fbfdf1d0cb2-cni-binary-copy\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:14.952012 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.951884 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-kubernetes\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.952012 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.951886 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjft5\" (UniqueName: \"kubernetes.io/projected/af327940-c861-4625-a6bb-1fbfdf1d0cb2-kube-api-access-qjft5\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:14.952012 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.951904 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-var-lib-cni-multus\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.952012 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.951926 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgm68\" (UniqueName: \"kubernetes.io/projected/94b49ea3-c88e-48e3-8fe3-6d9882628b2a-kube-api-access-dgm68\") pod \"iptables-alerter-jp9bl\" (UID: \"94b49ea3-c88e-48e3-8fe3-6d9882628b2a\") " pod="openshift-network-operator/iptables-alerter-jp9bl" Jul 09 09:54:14.952012 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.951956 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-sysctl-conf\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.952012 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952002 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-systemd\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952025 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-system-cni-dir\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952049 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-systemd-units\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952074 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-cni-netd\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952096 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/32dd5ea2-f421-45fd-b2c8-5e27535f407c-ovnkube-config\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952113 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-systemd\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952117 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-run-systemd\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952136 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-sysctl-conf\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952173 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-system-cni-dir\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952211 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h86td\" (UniqueName: \"kubernetes.io/projected/8edce7b1-4310-4adf-b754-3169b53b9a90-kube-api-access-h86td\") pod \"node-resolver-chwp7\" (UID: \"8edce7b1-4310-4adf-b754-3169b53b9a90\") " pod="openshift-dns/node-resolver-chwp7" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952236 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-etc-openvswitch\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952272 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952294 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952341 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.952404 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.952381 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952421 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-socket-dir\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:14.952456 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs podName:ab3cadc2-2f43-4d65-95a2-d83bae26ec1e nodeName:}" failed. No retries permitted until 2026-07-09 09:54:15.452433712 +0000 UTC m=+2.711383583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs") pod "network-metrics-daemon-7fx9w" (UID: "ab3cadc2-2f43-4d65-95a2-d83bae26ec1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952487 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-registration-dir\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952513 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-sys\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952537 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2ndk\" (UniqueName: \"kubernetes.io/projected/439a6e71-546c-4642-a628-15476b3deeeb-kube-api-access-j2ndk\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952561 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-multus-socket-dir-parent\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952563 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-registration-dir\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952629 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-sys\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952667 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-run-ovn-kubernetes\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952696 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952732 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/32dd5ea2-f421-45fd-b2c8-5e27535f407c-ovnkube-script-lib\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952736 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-multus-socket-dir-parent\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952799 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8edce7b1-4310-4adf-b754-3169b53b9a90-tmp-dir\") pod \"node-resolver-chwp7\" (UID: \"8edce7b1-4310-4adf-b754-3169b53b9a90\") " pod="openshift-dns/node-resolver-chwp7" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952838 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csjdn\" (UniqueName: \"kubernetes.io/projected/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-kube-api-access-csjdn\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952866 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5zv7\" (UniqueName: \"kubernetes.io/projected/b5ab5674-a77b-456f-a530-e47b0fb0e33c-kube-api-access-q5zv7\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.953050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.952922 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-sysconfig\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953057 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-os-release\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953074 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-sysconfig\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953099 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65dd7a41-8045-4903-bfbd-273242802bca-multus-daemon-config\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953120 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-kubelet\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953136 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af327940-c861-4625-a6bb-1fbfdf1d0cb2-os-release\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953159 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-run\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953157 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-os-release\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p744j\" (UniqueName: \"kubernetes.io/projected/8017af28-6540-46c3-9bc3-8b4ab3b83303-kube-api-access-p744j\") pod \"node-ca-grjk4\" (UID: \"8017af28-6540-46c3-9bc3-8b4ab3b83303\") " pod="openshift-image-registry/node-ca-grjk4" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953206 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-var-lib-kubelet\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953223 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-run-multus-certs\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953241 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/af327940-c861-4625-a6bb-1fbfdf1d0cb2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953240 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-run\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953268 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57g88\" (UniqueName: \"kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88\") pod \"network-check-target-fg9gs\" (UID: \"42a23145-ce54-4f13-856c-b112538017d0\") " pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953276 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-var-lib-kubelet\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953292 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-run-multus-certs\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953308 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8017af28-6540-46c3-9bc3-8b4ab3b83303-host\") pod \"node-ca-grjk4\" (UID: \"8017af28-6540-46c3-9bc3-8b4ab3b83303\") " pod="openshift-image-registry/node-ca-grjk4" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953320 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8017af28-6540-46c3-9bc3-8b4ab3b83303-host\") pod \"node-ca-grjk4\" (UID: \"8017af28-6540-46c3-9bc3-8b4ab3b83303\") " pod="openshift-image-registry/node-ca-grjk4" Jul 09 09:54:14.953679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953347 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-etc-kubernetes\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953378 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/af327940-c861-4625-a6bb-1fbfdf1d0cb2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953405 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-host\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953401 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-etc-kubernetes\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953429 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/439a6e71-546c-4642-a628-15476b3deeeb-etc-tuned\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953453 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-cnibin\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953460 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-host\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953487 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-run-netns\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953523 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9vbp\" (UniqueName: \"kubernetes.io/projected/32dd5ea2-f421-45fd-b2c8-5e27535f407c-kube-api-access-l9vbp\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953529 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-cnibin\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953549 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5e95a1c6-e119-456a-91d0-7887e896e8ae-agent-certs\") pod \"konnectivity-agent-v97nr\" (UID: \"5e95a1c6-e119-456a-91d0-7887e896e8ae\") " pod="kube-system/konnectivity-agent-v97nr" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953572 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5e95a1c6-e119-456a-91d0-7887e896e8ae-konnectivity-ca\") pod \"konnectivity-agent-v97nr\" (UID: \"5e95a1c6-e119-456a-91d0-7887e896e8ae\") " pod="kube-system/konnectivity-agent-v97nr" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953594 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-lib-modules\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953620 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-multus-conf-dir\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953594 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-socket-dir\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953683 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e0bd29c0-1d95-4097-9628-94da4639bcd5-dbus\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953703 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-multus-conf-dir\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.954555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953750 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-slash\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953756 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-lib-modules\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953749 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953785 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-sys-fs\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953810 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-modprobe-d\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953835 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-run-k8s-cni-cncf-io\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953877 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-run-k8s-cni-cncf-io\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953896 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-sys-fs\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953908 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-modprobe-d\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953936 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-run-ovn\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.953961 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-etc-selinux\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954014 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65dd7a41-8045-4903-bfbd-273242802bca-cni-binary-copy\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954079 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-run-netns\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954114 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-etc-selinux\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954120 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/32dd5ea2-f421-45fd-b2c8-5e27535f407c-env-overrides\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954155 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/32dd5ea2-f421-45fd-b2c8-5e27535f407c-ovn-node-metrics-cert\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954165 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-run-netns\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-sysctl-d\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.955316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954216 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/439a6e71-546c-4642-a628-15476b3deeeb-tmp\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954251 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8017af28-6540-46c3-9bc3-8b4ab3b83303-serviceca\") pod \"node-ca-grjk4\" (UID: \"8017af28-6540-46c3-9bc3-8b4ab3b83303\") " pod="openshift-image-registry/node-ca-grjk4" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954276 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-var-lib-cni-bin\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954310 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-etc-sysctl-d\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954416 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-267hw\" (UniqueName: \"kubernetes.io/projected/65dd7a41-8045-4903-bfbd-273242802bca-kube-api-access-267hw\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954461 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-host-var-lib-cni-bin\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954503 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65dd7a41-8045-4903-bfbd-273242802bca-cni-binary-copy\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954527 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65dd7a41-8045-4903-bfbd-273242802bca-multus-daemon-config\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954463 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-device-dir\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954664 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b5ab5674-a77b-456f-a530-e47b0fb0e33c-device-dir\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954748 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-multus-cni-dir\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954781 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954813 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-run-openvswitch\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954838 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-node-log\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954862 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-cni-bin\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954910 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-multus-cni-dir\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954923 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af327940-c861-4625-a6bb-1fbfdf1d0cb2-system-cni-dir\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:14.956297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954959 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8017af28-6540-46c3-9bc3-8b4ab3b83303-serviceca\") pod \"node-ca-grjk4\" (UID: \"8017af28-6540-46c3-9bc3-8b4ab3b83303\") " pod="openshift-image-registry/node-ca-grjk4" Jul 09 09:54:14.957159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.954976 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94b49ea3-c88e-48e3-8fe3-6d9882628b2a-host-slash\") pod \"iptables-alerter-jp9bl\" (UID: \"94b49ea3-c88e-48e3-8fe3-6d9882628b2a\") " pod="openshift-network-operator/iptables-alerter-jp9bl" Jul 09 09:54:14.957159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.955011 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-var-lib-kubelet\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.957159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.955064 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/439a6e71-546c-4642-a628-15476b3deeeb-var-lib-kubelet\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.957159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.955069 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-hostroot\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.957159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.955103 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e0bd29c0-1d95-4097-9628-94da4639bcd5-kubelet-config\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:14.957159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.955130 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-var-lib-openvswitch\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:14.957159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.955160 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8edce7b1-4310-4adf-b754-3169b53b9a90-hosts-file\") pod \"node-resolver-chwp7\" (UID: \"8edce7b1-4310-4adf-b754-3169b53b9a90\") " pod="openshift-dns/node-resolver-chwp7" Jul 09 09:54:14.957159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.955182 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af327940-c861-4625-a6bb-1fbfdf1d0cb2-cnibin\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:14.957159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.955205 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af327940-c861-4625-a6bb-1fbfdf1d0cb2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:14.957159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.955207 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65dd7a41-8045-4903-bfbd-273242802bca-hostroot\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.957159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.955227 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/94b49ea3-c88e-48e3-8fe3-6d9882628b2a-iptables-alerter-script\") pod \"iptables-alerter-jp9bl\" (UID: \"94b49ea3-c88e-48e3-8fe3-6d9882628b2a\") " pod="openshift-network-operator/iptables-alerter-jp9bl" Jul 09 09:54:14.957159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.957119 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/439a6e71-546c-4642-a628-15476b3deeeb-etc-tuned\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.957788 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.957164 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/439a6e71-546c-4642-a628-15476b3deeeb-tmp\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.964794 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.964772 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2ndk\" (UniqueName: \"kubernetes.io/projected/439a6e71-546c-4642-a628-15476b3deeeb-kube-api-access-j2ndk\") pod \"tuned-j68cp\" (UID: \"439a6e71-546c-4642-a628-15476b3deeeb\") " pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:14.964794 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.964787 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-267hw\" (UniqueName: \"kubernetes.io/projected/65dd7a41-8045-4903-bfbd-273242802bca-kube-api-access-267hw\") pod \"multus-bjdg8\" (UID: \"65dd7a41-8045-4903-bfbd-273242802bca\") " pod="openshift-multus/multus-bjdg8" Jul 09 09:54:14.964965 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.964948 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p744j\" (UniqueName: \"kubernetes.io/projected/8017af28-6540-46c3-9bc3-8b4ab3b83303-kube-api-access-p744j\") pod \"node-ca-grjk4\" (UID: \"8017af28-6540-46c3-9bc3-8b4ab3b83303\") " pod="openshift-image-registry/node-ca-grjk4" Jul 09 09:54:14.967105 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.967085 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5zv7\" (UniqueName: \"kubernetes.io/projected/b5ab5674-a77b-456f-a530-e47b0fb0e33c-kube-api-access-q5zv7\") pod \"aws-ebs-csi-driver-node-s5t8g\" (UID: \"b5ab5674-a77b-456f-a530-e47b0fb0e33c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:14.967733 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:14.967715 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csjdn\" (UniqueName: \"kubernetes.io/projected/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-kube-api-access-csjdn\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:15.056108 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056082 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:15.056284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056127 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-run-openvswitch\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.056284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056145 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-node-log\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.056284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056169 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-cni-bin\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.056284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056195 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af327940-c861-4625-a6bb-1fbfdf1d0cb2-system-cni-dir\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.056284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056219 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94b49ea3-c88e-48e3-8fe3-6d9882628b2a-host-slash\") pod \"iptables-alerter-jp9bl\" (UID: \"94b49ea3-c88e-48e3-8fe3-6d9882628b2a\") " pod="openshift-network-operator/iptables-alerter-jp9bl" Jul 09 09:54:15.056284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056224 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-run-openvswitch\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.056284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056239 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-node-log\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.056284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056233 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-cni-bin\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.056284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056246 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e0bd29c0-1d95-4097-9628-94da4639bcd5-kubelet-config\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:15.056284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056283 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/af327940-c861-4625-a6bb-1fbfdf1d0cb2-system-cni-dir\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056284 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/94b49ea3-c88e-48e3-8fe3-6d9882628b2a-host-slash\") pod \"iptables-alerter-jp9bl\" (UID: \"94b49ea3-c88e-48e3-8fe3-6d9882628b2a\") " pod="openshift-network-operator/iptables-alerter-jp9bl" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056315 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-var-lib-openvswitch\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.056251 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056338 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8edce7b1-4310-4adf-b754-3169b53b9a90-hosts-file\") pod \"node-resolver-chwp7\" (UID: \"8edce7b1-4310-4adf-b754-3169b53b9a90\") " pod="openshift-dns/node-resolver-chwp7" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056350 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e0bd29c0-1d95-4097-9628-94da4639bcd5-kubelet-config\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056361 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af327940-c861-4625-a6bb-1fbfdf1d0cb2-cnibin\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056384 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af327940-c861-4625-a6bb-1fbfdf1d0cb2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056369 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-var-lib-openvswitch\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.056399 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret podName:e0bd29c0-1d95-4097-9628-94da4639bcd5 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:15.556381041 +0000 UTC m=+2.815330885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret") pod "global-pull-secret-syncer-f987x" (UID: "e0bd29c0-1d95-4097-9628-94da4639bcd5") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056399 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8edce7b1-4310-4adf-b754-3169b53b9a90-hosts-file\") pod \"node-resolver-chwp7\" (UID: \"8edce7b1-4310-4adf-b754-3169b53b9a90\") " pod="openshift-dns/node-resolver-chwp7" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056431 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/94b49ea3-c88e-48e3-8fe3-6d9882628b2a-iptables-alerter-script\") pod \"iptables-alerter-jp9bl\" (UID: \"94b49ea3-c88e-48e3-8fe3-6d9882628b2a\") " pod="openshift-network-operator/iptables-alerter-jp9bl" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056444 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/af327940-c861-4625-a6bb-1fbfdf1d0cb2-cnibin\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056452 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-log-socket\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056506 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-log-socket\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056516 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/af327940-c861-4625-a6bb-1fbfdf1d0cb2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056534 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af327940-c861-4625-a6bb-1fbfdf1d0cb2-cni-binary-copy\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.056835 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjft5\" (UniqueName: \"kubernetes.io/projected/af327940-c861-4625-a6bb-1fbfdf1d0cb2-kube-api-access-qjft5\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056728 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgm68\" (UniqueName: \"kubernetes.io/projected/94b49ea3-c88e-48e3-8fe3-6d9882628b2a-kube-api-access-dgm68\") pod \"iptables-alerter-jp9bl\" (UID: \"94b49ea3-c88e-48e3-8fe3-6d9882628b2a\") " pod="openshift-network-operator/iptables-alerter-jp9bl" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056783 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-systemd-units\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056827 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-cni-netd\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056855 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/32dd5ea2-f421-45fd-b2c8-5e27535f407c-ovnkube-config\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056878 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-run-systemd\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056903 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h86td\" (UniqueName: \"kubernetes.io/projected/8edce7b1-4310-4adf-b754-3169b53b9a90-kube-api-access-h86td\") pod \"node-resolver-chwp7\" (UID: \"8edce7b1-4310-4adf-b754-3169b53b9a90\") " pod="openshift-dns/node-resolver-chwp7" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.056929 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-etc-openvswitch\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057008 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-run-ovn-kubernetes\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057018 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/94b49ea3-c88e-48e3-8fe3-6d9882628b2a-iptables-alerter-script\") pod \"iptables-alerter-jp9bl\" (UID: \"94b49ea3-c88e-48e3-8fe3-6d9882628b2a\") " pod="openshift-network-operator/iptables-alerter-jp9bl" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057040 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057069 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/32dd5ea2-f421-45fd-b2c8-5e27535f407c-ovnkube-script-lib\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057094 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-run-systemd\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057094 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8edce7b1-4310-4adf-b754-3169b53b9a90-tmp-dir\") pod \"node-resolver-chwp7\" (UID: \"8edce7b1-4310-4adf-b754-3169b53b9a90\") " pod="openshift-dns/node-resolver-chwp7" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057158 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-kubelet\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057191 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af327940-c861-4625-a6bb-1fbfdf1d0cb2-os-release\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057235 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/af327940-c861-4625-a6bb-1fbfdf1d0cb2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.057755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057273 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57g88\" (UniqueName: \"kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88\") pod \"network-check-target-fg9gs\" (UID: \"42a23145-ce54-4f13-856c-b112538017d0\") " pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057311 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/af327940-c861-4625-a6bb-1fbfdf1d0cb2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057349 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-run-netns\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057353 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8edce7b1-4310-4adf-b754-3169b53b9a90-tmp-dir\") pod \"node-resolver-chwp7\" (UID: \"8edce7b1-4310-4adf-b754-3169b53b9a90\") " pod="openshift-dns/node-resolver-chwp7" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057392 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057418 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-systemd-units\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057419 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9vbp\" (UniqueName: \"kubernetes.io/projected/32dd5ea2-f421-45fd-b2c8-5e27535f407c-kube-api-access-l9vbp\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057469 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5e95a1c6-e119-456a-91d0-7887e896e8ae-agent-certs\") pod \"konnectivity-agent-v97nr\" (UID: \"5e95a1c6-e119-456a-91d0-7887e896e8ae\") " pod="kube-system/konnectivity-agent-v97nr" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057493 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-etc-openvswitch\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057508 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5e95a1c6-e119-456a-91d0-7887e896e8ae-konnectivity-ca\") pod \"konnectivity-agent-v97nr\" (UID: \"5e95a1c6-e119-456a-91d0-7887e896e8ae\") " pod="kube-system/konnectivity-agent-v97nr" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057557 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-kubelet\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.058405 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-cni-netd\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.058375 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/af327940-c861-4625-a6bb-1fbfdf1d0cb2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.058385 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/32dd5ea2-f421-45fd-b2c8-5e27535f407c-ovnkube-config\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.058399 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e0bd29c0-1d95-4097-9628-94da4639bcd5-dbus\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.058476 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-slash\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.058531 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/af327940-c861-4625-a6bb-1fbfdf1d0cb2-os-release\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.058612 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.058535 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-run-ovn\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.059504 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.058580 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-run-ovn\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.059504 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.058592 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/32dd5ea2-f421-45fd-b2c8-5e27535f407c-env-overrides\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.059504 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.058597 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/32dd5ea2-f421-45fd-b2c8-5e27535f407c-ovnkube-script-lib\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.059504 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.058629 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/32dd5ea2-f421-45fd-b2c8-5e27535f407c-ovn-node-metrics-cert\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.059504 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.058629 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-slash\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.059504 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057188 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/af327940-c861-4625-a6bb-1fbfdf1d0cb2-cni-binary-copy\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.059504 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.059050 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/32dd5ea2-f421-45fd-b2c8-5e27535f407c-env-overrides\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.059504 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.057236 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-run-ovn-kubernetes\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.059504 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.059143 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/af327940-c861-4625-a6bb-1fbfdf1d0cb2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.059504 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.059145 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/32dd5ea2-f421-45fd-b2c8-5e27535f407c-host-run-netns\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.060060 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.059559 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e0bd29c0-1d95-4097-9628-94da4639bcd5-dbus\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:15.060060 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.059706 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5e95a1c6-e119-456a-91d0-7887e896e8ae-konnectivity-ca\") pod \"konnectivity-agent-v97nr\" (UID: \"5e95a1c6-e119-456a-91d0-7887e896e8ae\") " pod="kube-system/konnectivity-agent-v97nr" Jul 09 09:54:15.062283 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.062252 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5e95a1c6-e119-456a-91d0-7887e896e8ae-agent-certs\") pod \"konnectivity-agent-v97nr\" (UID: \"5e95a1c6-e119-456a-91d0-7887e896e8ae\") " pod="kube-system/konnectivity-agent-v97nr" Jul 09 09:54:15.063143 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.063121 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/32dd5ea2-f421-45fd-b2c8-5e27535f407c-ovn-node-metrics-cert\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.069593 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.069565 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:15.069732 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.069599 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:15.069732 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.069617 2582 projected.go:196] Error preparing data for projected volume kube-api-access-57g88 for pod openshift-network-diagnostics/network-check-target-fg9gs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:15.069732 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.069721 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88 podName:42a23145-ce54-4f13-856c-b112538017d0 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:15.569701494 +0000 UTC m=+2.828651354 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-57g88" (UniqueName: "kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88") pod "network-check-target-fg9gs" (UID: "42a23145-ce54-4f13-856c-b112538017d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:15.070539 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.070513 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjft5\" (UniqueName: \"kubernetes.io/projected/af327940-c861-4625-a6bb-1fbfdf1d0cb2-kube-api-access-qjft5\") pod \"multus-additional-cni-plugins-nktqq\" (UID: \"af327940-c861-4625-a6bb-1fbfdf1d0cb2\") " pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.071821 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.071800 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h86td\" (UniqueName: \"kubernetes.io/projected/8edce7b1-4310-4adf-b754-3169b53b9a90-kube-api-access-h86td\") pod \"node-resolver-chwp7\" (UID: \"8edce7b1-4310-4adf-b754-3169b53b9a90\") " pod="openshift-dns/node-resolver-chwp7" Jul 09 09:54:15.071916 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.071836 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgm68\" (UniqueName: \"kubernetes.io/projected/94b49ea3-c88e-48e3-8fe3-6d9882628b2a-kube-api-access-dgm68\") pod \"iptables-alerter-jp9bl\" (UID: \"94b49ea3-c88e-48e3-8fe3-6d9882628b2a\") " pod="openshift-network-operator/iptables-alerter-jp9bl" Jul 09 09:54:15.072127 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.072108 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9vbp\" (UniqueName: \"kubernetes.io/projected/32dd5ea2-f421-45fd-b2c8-5e27535f407c-kube-api-access-l9vbp\") pod \"ovnkube-node-psrgx\" (UID: \"32dd5ea2-f421-45fd-b2c8-5e27535f407c\") " pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.162549 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.162521 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" Jul 09 09:54:15.177118 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:15.177082 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a90544793a6b2ed8c167388511c8cc.slice/crio-a18d8e70c3e1e65d691bedde6c1d1dca4b3841f48e1ae596d4d749bb8ffbaca7 WatchSource:0}: Error finding container a18d8e70c3e1e65d691bedde6c1d1dca4b3841f48e1ae596d4d749bb8ffbaca7: Status 404 returned error can't find the container with id a18d8e70c3e1e65d691bedde6c1d1dca4b3841f48e1ae596d4d749bb8ffbaca7 Jul 09 09:54:15.180492 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:15.180471 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4780b8f2df1828ad4bb2a95b7e56dc50.slice/crio-7d7c640e0cce9689f6827beedc289b87b825c5ee05b7055f828fd139f6b021f5 WatchSource:0}: Error finding container 7d7c640e0cce9689f6827beedc289b87b825c5ee05b7055f828fd139f6b021f5: Status 404 returned error can't find the container with id 7d7c640e0cce9689f6827beedc289b87b825c5ee05b7055f828fd139f6b021f5 Jul 09 09:54:15.180958 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:15.180928 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5ab5674_a77b_456f_a530_e47b0fb0e33c.slice/crio-b2e16d869f78b3f47599461386e8dfa40d9270984aa8af484d612d010ba45a6c WatchSource:0}: Error finding container b2e16d869f78b3f47599461386e8dfa40d9270984aa8af484d612d010ba45a6c: Status 404 returned error can't find the container with id b2e16d869f78b3f47599461386e8dfa40d9270984aa8af484d612d010ba45a6c Jul 09 09:54:15.183373 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.183362 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 09:54:15.186303 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.186162 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j68cp" Jul 09 09:54:15.195679 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:15.195661 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod439a6e71_546c_4642_a628_15476b3deeeb.slice/crio-9ec0071560cf342c53208cdcd11febe8f3cd602fa981e002632ebf05ffb9296f WatchSource:0}: Error finding container 9ec0071560cf342c53208cdcd11febe8f3cd602fa981e002632ebf05ffb9296f: Status 404 returned error can't find the container with id 9ec0071560cf342c53208cdcd11febe8f3cd602fa981e002632ebf05ffb9296f Jul 09 09:54:15.203535 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.203516 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-grjk4" Jul 09 09:54:15.211525 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:15.211504 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8017af28_6540_46c3_9bc3_8b4ab3b83303.slice/crio-ccfbd0abc534adbd37b1dcf19438317f11c1b893fbd705c85726fb48f491f9aa WatchSource:0}: Error finding container ccfbd0abc534adbd37b1dcf19438317f11c1b893fbd705c85726fb48f491f9aa: Status 404 returned error can't find the container with id ccfbd0abc534adbd37b1dcf19438317f11c1b893fbd705c85726fb48f491f9aa Jul 09 09:54:15.217757 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.217731 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bjdg8" Jul 09 09:54:15.225842 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:15.225821 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65dd7a41_8045_4903_bfbd_273242802bca.slice/crio-223ae6cf00966223e441e7fc310cfad761ad8aca307dc496fbc1af07852ef431 WatchSource:0}: Error finding container 223ae6cf00966223e441e7fc310cfad761ad8aca307dc496fbc1af07852ef431: Status 404 returned error can't find the container with id 223ae6cf00966223e441e7fc310cfad761ad8aca307dc496fbc1af07852ef431 Jul 09 09:54:15.237555 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.237538 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:15.245756 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:15.245735 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32dd5ea2_f421_45fd_b2c8_5e27535f407c.slice/crio-b1fc60cace5d90f50b829fbb9a2b3a21f1208983da5e535520631d7f6fcfce75 WatchSource:0}: Error finding container b1fc60cace5d90f50b829fbb9a2b3a21f1208983da5e535520631d7f6fcfce75: Status 404 returned error can't find the container with id b1fc60cace5d90f50b829fbb9a2b3a21f1208983da5e535520631d7f6fcfce75 Jul 09 09:54:15.254786 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.254770 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v97nr" Jul 09 09:54:15.263261 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:15.263241 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e95a1c6_e119_456a_91d0_7887e896e8ae.slice/crio-cb89292ed1e666ec8f7dfc7a522652dc76a84027d2f073d2b62cd1e15663f5ff WatchSource:0}: Error finding container cb89292ed1e666ec8f7dfc7a522652dc76a84027d2f073d2b62cd1e15663f5ff: Status 404 returned error can't find the container with id cb89292ed1e666ec8f7dfc7a522652dc76a84027d2f073d2b62cd1e15663f5ff Jul 09 09:54:15.276908 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.276891 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-chwp7" Jul 09 09:54:15.285227 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:15.285203 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8edce7b1_4310_4adf_b754_3169b53b9a90.slice/crio-6feb4cdf58884c953f9c1a772ef0f9e7dcf1779c78914bafc90bb068057c21e7 WatchSource:0}: Error finding container 6feb4cdf58884c953f9c1a772ef0f9e7dcf1779c78914bafc90bb068057c21e7: Status 404 returned error can't find the container with id 6feb4cdf58884c953f9c1a772ef0f9e7dcf1779c78914bafc90bb068057c21e7 Jul 09 09:54:15.288213 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.288197 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nktqq" Jul 09 09:54:15.296228 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.296210 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jp9bl" Jul 09 09:54:15.297014 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:15.296997 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf327940_c861_4625_a6bb_1fbfdf1d0cb2.slice/crio-d893fe4df402d768b135370a254ba2acb75cf9a1ea27e3b5be1ef05ab6ef7b4e WatchSource:0}: Error finding container d893fe4df402d768b135370a254ba2acb75cf9a1ea27e3b5be1ef05ab6ef7b4e: Status 404 returned error can't find the container with id d893fe4df402d768b135370a254ba2acb75cf9a1ea27e3b5be1ef05ab6ef7b4e Jul 09 09:54:15.305467 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:15.305446 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b49ea3_c88e_48e3_8fe3_6d9882628b2a.slice/crio-3d2b5e6db2f7b70d4607149fce66acff03ad1a7bbd0413cf14f60cb1009be7bb WatchSource:0}: Error finding container 3d2b5e6db2f7b70d4607149fce66acff03ad1a7bbd0413cf14f60cb1009be7bb: Status 404 returned error can't find the container with id 3d2b5e6db2f7b70d4607149fce66acff03ad1a7bbd0413cf14f60cb1009be7bb Jul 09 09:54:15.307182 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.307164 2582 kuberuntime_manager.go:1459] "Unhandled Error" err="container iptables-alerter start failed in pod iptables-alerter-jp9bl_openshift-network-operator(94b49ea3-c88e-48e3-8fe3-6d9882628b2a): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 09 09:54:15.307233 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.307198 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-jp9bl" podUID="94b49ea3-c88e-48e3-8fe3-6d9882628b2a" Jul 09 09:54:15.462392 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.462361 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:15.462580 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.462533 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:15.462670 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.462607 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs podName:ab3cadc2-2f43-4d65-95a2-d83bae26ec1e nodeName:}" failed. No retries permitted until 2026-07-09 09:54:16.462586642 +0000 UTC m=+3.721536483 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs") pod "network-metrics-daemon-7fx9w" (UID: "ab3cadc2-2f43-4d65-95a2-d83bae26ec1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:15.562880 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.562795 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:15.563040 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.562939 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:15.563040 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.563007 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret podName:e0bd29c0-1d95-4097-9628-94da4639bcd5 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:16.562987607 +0000 UTC m=+3.821937465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret") pod "global-pull-secret-syncer-f987x" (UID: "e0bd29c0-1d95-4097-9628-94da4639bcd5") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:15.664223 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.664184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57g88\" (UniqueName: \"kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88\") pod \"network-check-target-fg9gs\" (UID: \"42a23145-ce54-4f13-856c-b112538017d0\") " pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:15.664468 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.664447 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:15.664561 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.664476 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:15.664561 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.664490 2582 projected.go:196] Error preparing data for projected volume kube-api-access-57g88 for pod openshift-network-diagnostics/network-check-target-fg9gs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:15.664702 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:15.664575 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88 podName:42a23145-ce54-4f13-856c-b112538017d0 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:16.664554552 +0000 UTC m=+3.923504395 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-57g88" (UniqueName: "kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88") pod "network-check-target-fg9gs" (UID: "42a23145-ce54-4f13-856c-b112538017d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:15.881266 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.881187 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:15.898053 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.897990 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 09:49:15 +0000 UTC" deadline="2028-04-07 07:22:22.444122052 +0000 UTC" Jul 09 09:54:15.898053 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:15.898051 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15309h28m6.546075606s" Jul 09 09:54:16.059864 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.059799 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" event={"ID":"b5ab5674-a77b-456f-a530-e47b0fb0e33c","Type":"ContainerStarted","Data":"b2e16d869f78b3f47599461386e8dfa40d9270984aa8af484d612d010ba45a6c"} Jul 09 09:54:16.074714 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.074659 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-chwp7" event={"ID":"8edce7b1-4310-4adf-b754-3169b53b9a90","Type":"ContainerStarted","Data":"6feb4cdf58884c953f9c1a772ef0f9e7dcf1779c78914bafc90bb068057c21e7"} Jul 09 09:54:16.092361 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.092326 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v97nr" event={"ID":"5e95a1c6-e119-456a-91d0-7887e896e8ae","Type":"ContainerStarted","Data":"cb89292ed1e666ec8f7dfc7a522652dc76a84027d2f073d2b62cd1e15663f5ff"} Jul 09 09:54:16.102780 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.102751 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" event={"ID":"32dd5ea2-f421-45fd-b2c8-5e27535f407c","Type":"ContainerStarted","Data":"b1fc60cace5d90f50b829fbb9a2b3a21f1208983da5e535520631d7f6fcfce75"} Jul 09 09:54:16.119778 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.119749 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bjdg8" event={"ID":"65dd7a41-8045-4903-bfbd-273242802bca","Type":"ContainerStarted","Data":"223ae6cf00966223e441e7fc310cfad761ad8aca307dc496fbc1af07852ef431"} Jul 09 09:54:16.129049 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.129013 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j68cp" event={"ID":"439a6e71-546c-4642-a628-15476b3deeeb","Type":"ContainerStarted","Data":"9ec0071560cf342c53208cdcd11febe8f3cd602fa981e002632ebf05ffb9296f"} Jul 09 09:54:16.136569 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.136491 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" event={"ID":"4780b8f2df1828ad4bb2a95b7e56dc50","Type":"ContainerStarted","Data":"7d7c640e0cce9689f6827beedc289b87b825c5ee05b7055f828fd139f6b021f5"} Jul 09 09:54:16.141827 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.141799 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" event={"ID":"18a90544793a6b2ed8c167388511c8cc","Type":"ContainerStarted","Data":"a18d8e70c3e1e65d691bedde6c1d1dca4b3841f48e1ae596d4d749bb8ffbaca7"} Jul 09 09:54:16.161456 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.160248 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jp9bl" event={"ID":"94b49ea3-c88e-48e3-8fe3-6d9882628b2a","Type":"ContainerStarted","Data":"3d2b5e6db2f7b70d4607149fce66acff03ad1a7bbd0413cf14f60cb1009be7bb"} Jul 09 09:54:16.164220 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:16.163951 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"iptables-alerter\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:28389dd118629aa9d7c2525202f3ad6359afb98b6a3bf7672cb1ecfae2418e23\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-network-operator/iptables-alerter-jp9bl" podUID="94b49ea3-c88e-48e3-8fe3-6d9882628b2a" Jul 09 09:54:16.180967 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.178706 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nktqq" event={"ID":"af327940-c861-4625-a6bb-1fbfdf1d0cb2","Type":"ContainerStarted","Data":"d893fe4df402d768b135370a254ba2acb75cf9a1ea27e3b5be1ef05ab6ef7b4e"} Jul 09 09:54:16.185096 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.185073 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:16.200999 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.200969 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-grjk4" event={"ID":"8017af28-6540-46c3-9bc3-8b4ab3b83303","Type":"ContainerStarted","Data":"ccfbd0abc534adbd37b1dcf19438317f11c1b893fbd705c85726fb48f491f9aa"} Jul 09 09:54:16.468577 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.468485 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 09:54:16.472718 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.472687 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:16.472876 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:16.472855 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:16.472952 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:16.472918 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs podName:ab3cadc2-2f43-4d65-95a2-d83bae26ec1e nodeName:}" failed. No retries permitted until 2026-07-09 09:54:18.472898305 +0000 UTC m=+5.731848159 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs") pod "network-metrics-daemon-7fx9w" (UID: "ab3cadc2-2f43-4d65-95a2-d83bae26ec1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:16.573359 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.573260 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:16.573561 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:16.573506 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:16.573639 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:16.573579 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret podName:e0bd29c0-1d95-4097-9628-94da4639bcd5 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:18.573557639 +0000 UTC m=+5.832507481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret") pod "global-pull-secret-syncer-f987x" (UID: "e0bd29c0-1d95-4097-9628-94da4639bcd5") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:16.674309 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.674267 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57g88\" (UniqueName: \"kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88\") pod \"network-check-target-fg9gs\" (UID: \"42a23145-ce54-4f13-856c-b112538017d0\") " pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:16.674503 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:16.674463 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:16.674503 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:16.674488 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:16.674503 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:16.674503 2582 projected.go:196] Error preparing data for projected volume kube-api-access-57g88 for pod openshift-network-diagnostics/network-check-target-fg9gs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:16.674680 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:16.674567 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88 podName:42a23145-ce54-4f13-856c-b112538017d0 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:18.674547008 +0000 UTC m=+5.933496847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-57g88" (UniqueName: "kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88") pod "network-check-target-fg9gs" (UID: "42a23145-ce54-4f13-856c-b112538017d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:16.899230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.898906 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 09:49:15 +0000 UTC" deadline="2028-04-25 04:53:27.138909726 +0000 UTC" Jul 09 09:54:16.899230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:16.899048 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15738h59m10.2398665s" Jul 09 09:54:17.034957 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:17.034912 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:17.035150 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:17.035055 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:17.035518 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:17.035464 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:17.035634 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:17.035564 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:17.035748 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:17.035727 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:17.035883 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:17.035851 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:18.491027 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:18.490890 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:18.491661 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:18.491128 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:18.491661 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:18.491215 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs podName:ab3cadc2-2f43-4d65-95a2-d83bae26ec1e nodeName:}" failed. No retries permitted until 2026-07-09 09:54:22.491190851 +0000 UTC m=+9.750140694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs") pod "network-metrics-daemon-7fx9w" (UID: "ab3cadc2-2f43-4d65-95a2-d83bae26ec1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:18.592734 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:18.592154 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:18.592734 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:18.592343 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:18.592734 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:18.592433 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret podName:e0bd29c0-1d95-4097-9628-94da4639bcd5 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:22.592410839 +0000 UTC m=+9.851360690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret") pod "global-pull-secret-syncer-f987x" (UID: "e0bd29c0-1d95-4097-9628-94da4639bcd5") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:18.693526 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:18.692956 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57g88\" (UniqueName: \"kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88\") pod \"network-check-target-fg9gs\" (UID: \"42a23145-ce54-4f13-856c-b112538017d0\") " pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:18.693526 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:18.693083 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:18.693526 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:18.693100 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:18.693526 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:18.693109 2582 projected.go:196] Error preparing data for projected volume kube-api-access-57g88 for pod openshift-network-diagnostics/network-check-target-fg9gs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:18.693526 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:18.693168 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88 podName:42a23145-ce54-4f13-856c-b112538017d0 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:22.693154564 +0000 UTC m=+9.952104403 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-57g88" (UniqueName: "kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88") pod "network-check-target-fg9gs" (UID: "42a23145-ce54-4f13-856c-b112538017d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:19.035518 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:19.035480 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:19.035518 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:19.035517 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:19.035789 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:19.035483 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:19.035789 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:19.035632 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:19.035789 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:19.035758 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:19.035907 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:19.035840 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:21.035048 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:21.035017 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:21.035464 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:21.035132 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:21.035464 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:21.035140 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:21.035464 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:21.035262 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:21.035464 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:21.035335 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:21.035464 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:21.035427 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:22.528181 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:22.528145 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:22.528675 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:22.528333 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:22.528675 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:22.528412 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs podName:ab3cadc2-2f43-4d65-95a2-d83bae26ec1e nodeName:}" failed. No retries permitted until 2026-07-09 09:54:30.528388019 +0000 UTC m=+17.787337864 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs") pod "network-metrics-daemon-7fx9w" (UID: "ab3cadc2-2f43-4d65-95a2-d83bae26ec1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:22.629626 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:22.629568 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:22.629832 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:22.629741 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:22.629832 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:22.629814 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret podName:e0bd29c0-1d95-4097-9628-94da4639bcd5 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:30.629795433 +0000 UTC m=+17.888745294 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret") pod "global-pull-secret-syncer-f987x" (UID: "e0bd29c0-1d95-4097-9628-94da4639bcd5") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:22.730253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:22.730202 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57g88\" (UniqueName: \"kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88\") pod \"network-check-target-fg9gs\" (UID: \"42a23145-ce54-4f13-856c-b112538017d0\") " pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:22.730442 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:22.730411 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:22.730442 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:22.730430 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:22.730442 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:22.730443 2582 projected.go:196] Error preparing data for projected volume kube-api-access-57g88 for pod openshift-network-diagnostics/network-check-target-fg9gs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:22.730611 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:22.730505 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88 podName:42a23145-ce54-4f13-856c-b112538017d0 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:30.730483576 +0000 UTC m=+17.989433432 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-57g88" (UniqueName: "kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88") pod "network-check-target-fg9gs" (UID: "42a23145-ce54-4f13-856c-b112538017d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:23.035499 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:23.035470 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:23.035724 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:23.035475 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:23.035724 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:23.035584 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:23.035724 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:23.035677 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:23.035724 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:23.035475 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:23.035969 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:23.035792 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:25.035948 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:25.035076 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:25.035948 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:25.035205 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:25.035948 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:25.035604 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:25.035948 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:25.035713 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:25.035948 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:25.035778 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:25.035948 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:25.035861 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:27.034783 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:27.034741 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:27.034783 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:27.034762 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:27.035220 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:27.034854 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:27.035220 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:27.034859 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:27.035220 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:27.034929 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:27.035220 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:27.035095 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:29.035238 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:29.035201 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:29.035238 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:29.035226 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:29.035793 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:29.035351 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:29.035793 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:29.035362 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:29.035793 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:29.035463 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:29.035793 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:29.035585 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:30.592984 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:30.592950 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:30.593435 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:30.593085 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:30.593435 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:30.593155 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs podName:ab3cadc2-2f43-4d65-95a2-d83bae26ec1e nodeName:}" failed. No retries permitted until 2026-07-09 09:54:46.593134688 +0000 UTC m=+33.852084548 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs") pod "network-metrics-daemon-7fx9w" (UID: "ab3cadc2-2f43-4d65-95a2-d83bae26ec1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:30.693866 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:30.693827 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:30.694054 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:30.693983 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:30.694122 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:30.694056 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret podName:e0bd29c0-1d95-4097-9628-94da4639bcd5 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:46.694037274 +0000 UTC m=+33.952987130 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret") pod "global-pull-secret-syncer-f987x" (UID: "e0bd29c0-1d95-4097-9628-94da4639bcd5") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:30.795222 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:30.795184 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57g88\" (UniqueName: \"kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88\") pod \"network-check-target-fg9gs\" (UID: \"42a23145-ce54-4f13-856c-b112538017d0\") " pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:30.795450 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:30.795372 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:30.795450 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:30.795398 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:30.795450 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:30.795411 2582 projected.go:196] Error preparing data for projected volume kube-api-access-57g88 for pod openshift-network-diagnostics/network-check-target-fg9gs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:30.795604 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:30.795476 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88 podName:42a23145-ce54-4f13-856c-b112538017d0 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:46.795457671 +0000 UTC m=+34.054407518 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-57g88" (UniqueName: "kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88") pod "network-check-target-fg9gs" (UID: "42a23145-ce54-4f13-856c-b112538017d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:31.035309 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:31.035269 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:31.035530 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:31.035272 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:31.035530 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:31.035402 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:31.035530 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:31.035400 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:31.035530 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:31.035470 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:31.035733 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:31.035568 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:33.034599 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:33.034566 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:33.035035 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:33.034566 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:33.035035 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:33.034566 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:33.035035 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:33.034720 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:33.035035 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:33.034811 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:33.035035 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:33.034854 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:35.034516 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:35.034469 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:35.034516 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:35.034501 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:35.035042 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:35.034501 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:35.035042 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:35.034575 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:35.035042 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:35.034687 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:35.035042 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:35.034755 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:36.249988 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:36.249534 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" event={"ID":"32dd5ea2-f421-45fd-b2c8-5e27535f407c","Type":"ContainerStarted","Data":"7a9791f3ba495397acb6a4a7cf433b94027df0d1088167b18d52f2b51a7252fd"} Jul 09 09:54:36.250538 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:36.250082 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" event={"ID":"32dd5ea2-f421-45fd-b2c8-5e27535f407c","Type":"ContainerStarted","Data":"98ea78b2d9ae684c153e5094130532993cda325ff8a4b00ffeafef507cc1e3ef"} Jul 09 09:54:36.250538 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:36.250140 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" event={"ID":"32dd5ea2-f421-45fd-b2c8-5e27535f407c","Type":"ContainerStarted","Data":"ac9515b655efba97e565b315ce34ed10f977951cbe90549a6db645c3821a3f5d"} Jul 09 09:54:36.250876 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:36.250857 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bjdg8" event={"ID":"65dd7a41-8045-4903-bfbd-273242802bca","Type":"ContainerStarted","Data":"07b262c024c4cf5b90c57e96ca5dfae07e8a3044867e4c3806c180683027d17e"} Jul 09 09:54:36.252078 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:36.252057 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j68cp" event={"ID":"439a6e71-546c-4642-a628-15476b3deeeb","Type":"ContainerStarted","Data":"99c71493db3afa102465accfdf62d1e7e0a7a5da2db8169d3fefc9889a407fcc"} Jul 09 09:54:36.253190 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:36.253168 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" event={"ID":"4780b8f2df1828ad4bb2a95b7e56dc50","Type":"ContainerStarted","Data":"d8a0ebcdec821a554336664436232d1e018c8c105b243b2eb5ee41f1e2fd0581"} Jul 09 09:54:36.263515 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:36.263423 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" Jul 09 09:54:36.273593 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:36.273575 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 09:54:36.274336 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:36.274317 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal"] Jul 09 09:54:36.282878 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:36.282842 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j68cp" podStartSLOduration=1.867091034 podStartE2EDuration="22.282829949s" totalImagesPullingTime="20.415738915s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:36.28231765 +0000 UTC m=+23.541267530" watchObservedRunningTime="2026-07-09 09:54:36.282829949 +0000 UTC m=+23.541779844" Jul 09 09:54:36.298330 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:36.298293 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" podStartSLOduration=0.298282642 podStartE2EDuration="298.282642ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:36.298170156 +0000 UTC m=+23.557120016" watchObservedRunningTime="2026-07-09 09:54:36.298282642 +0000 UTC m=+23.557232500" Jul 09 09:54:36.315863 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:36.315829 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bjdg8" podStartSLOduration=1.894419118 podStartE2EDuration="22.315817127s" totalImagesPullingTime="20.421398009s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:36.315590256 +0000 UTC m=+23.574540114" watchObservedRunningTime="2026-07-09 09:54:36.315817127 +0000 UTC m=+23.574766987" Jul 09 09:54:37.035340 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.035309 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:37.035573 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.035344 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:37.035573 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.035407 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:37.035573 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:37.035419 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:37.035573 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:37.035477 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:37.035573 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:37.035521 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:37.256675 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.256644 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jp9bl" event={"ID":"94b49ea3-c88e-48e3-8fe3-6d9882628b2a","Type":"ContainerStarted","Data":"2a28b1c34c3ca5f2fd528f97477842f5c1cf4950ed06a9bf1634391490677284"} Jul 09 09:54:37.258230 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.258208 2582 generic.go:356] "Generic (PLEG): container finished" podID="af327940-c861-4625-a6bb-1fbfdf1d0cb2" containerID="4146186b91a5208ea55f6404d9be4a943efa1abd36b7d081d8f1c17b411c0dba" exitCode=0 Jul 09 09:54:37.258334 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.258274 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nktqq" event={"ID":"af327940-c861-4625-a6bb-1fbfdf1d0cb2","Type":"ContainerDied","Data":"4146186b91a5208ea55f6404d9be4a943efa1abd36b7d081d8f1c17b411c0dba"} Jul 09 09:54:37.259803 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.259760 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-grjk4" event={"ID":"8017af28-6540-46c3-9bc3-8b4ab3b83303","Type":"ContainerStarted","Data":"38cd453a267348b8759d9fce36cf33b79d833220a0aad0ef1484bdb774ea8c5e"} Jul 09 09:54:37.261147 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.261121 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" event={"ID":"b5ab5674-a77b-456f-a530-e47b0fb0e33c","Type":"ContainerStarted","Data":"6fb48cdc6d6f2e592564a8937685efc35b599f35c03f4f2fc02f8510fa3324fc"} Jul 09 09:54:37.262567 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.262547 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-chwp7" event={"ID":"8edce7b1-4310-4adf-b754-3169b53b9a90","Type":"ContainerStarted","Data":"3c0f344ab5a1d97e9a58bc5b374823d481c8ecb69bc6683522e9ab0e5edf9e15"} Jul 09 09:54:37.263913 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.263893 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v97nr" event={"ID":"5e95a1c6-e119-456a-91d0-7887e896e8ae","Type":"ContainerStarted","Data":"85b7ae2c2bca019cc7dc852bbc8c9bfaf2e24657cecb86900f143b75c3cbf765"} Jul 09 09:54:37.266546 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.266529 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" event={"ID":"32dd5ea2-f421-45fd-b2c8-5e27535f407c","Type":"ContainerStarted","Data":"b1c151396fc2b76c22e49b910ea307cb5b6b0434b074424eae40fbf2f42ae062"} Jul 09 09:54:37.266637 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.266587 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" event={"ID":"32dd5ea2-f421-45fd-b2c8-5e27535f407c","Type":"ContainerStarted","Data":"ea5c6412069a939bc40f33818b9cb19ae9d6bf7e53a17980b1be971fec598d31"} Jul 09 09:54:37.266682 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.266650 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" event={"ID":"32dd5ea2-f421-45fd-b2c8-5e27535f407c","Type":"ContainerStarted","Data":"ee8dcffc5578120585c9e596073d92db63a97cae691044a765a4afb171653af6"} Jul 09 09:54:37.269871 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.269848 2582 generic.go:356] "Generic (PLEG): container finished" podID="18a90544793a6b2ed8c167388511c8cc" containerID="7562b05f8a8d180339e3d36016e0359ed0cfbbd2a216a94c94352f89578b8712" exitCode=0 Jul 09 09:54:37.270267 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.269942 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" event={"ID":"18a90544793a6b2ed8c167388511c8cc","Type":"ContainerDied","Data":"7562b05f8a8d180339e3d36016e0359ed0cfbbd2a216a94c94352f89578b8712"} Jul 09 09:54:37.272726 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.271063 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" Jul 09 09:54:37.276429 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.276395 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jp9bl" podStartSLOduration=2.872965372 podStartE2EDuration="23.276384538s" totalImagesPullingTime="20.403419166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-09 09:54:37.27582732 +0000 UTC m=+24.534777179" watchObservedRunningTime="2026-07-09 09:54:37.276384538 +0000 UTC m=+24.535334397" Jul 09 09:54:37.285489 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.285473 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 09:54:37.285573 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:37.285511 2582 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-220.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-220.ec2.internal" Jul 09 09:54:37.324096 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.323831 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-grjk4" podStartSLOduration=2.925928771 podStartE2EDuration="23.323815382s" totalImagesPullingTime="20.397886611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:37.322821487 +0000 UTC m=+24.581771347" watchObservedRunningTime="2026-07-09 09:54:37.323815382 +0000 UTC m=+24.582765242" Jul 09 09:54:37.342722 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.342685 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v97nr" podStartSLOduration=2.996592644 podStartE2EDuration="23.3426724s" totalImagesPullingTime="20.346079756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:37.342499464 +0000 UTC m=+24.601449320" watchObservedRunningTime="2026-07-09 09:54:37.3426724 +0000 UTC m=+24.601622258" Jul 09 09:54:37.386517 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:37.386473 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-chwp7" podStartSLOduration=3.072868429 podStartE2EDuration="23.386457691s" totalImagesPullingTime="20.313589262s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:37.36828045 +0000 UTC m=+24.627230311" watchObservedRunningTime="2026-07-09 09:54:37.386457691 +0000 UTC m=+24.645407575" Jul 09 09:54:38.083936 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:38.083915 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 09:54:38.274148 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:38.274065 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" event={"ID":"18a90544793a6b2ed8c167388511c8cc","Type":"ContainerStarted","Data":"c82a3f0639f265697a9dddb25260db7b70bf9d05155fdbc32b7cea3dd8858664"} Jul 09 09:54:38.275871 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:38.275844 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" event={"ID":"b5ab5674-a77b-456f-a530-e47b0fb0e33c","Type":"ContainerStarted","Data":"abfc1a931211415a7978bc77dda023d0258846c402a88395404d5bc12b15c22c"} Jul 09 09:54:38.293733 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:38.293692 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-220.ec2.internal" podStartSLOduration=23.293678105 podStartE2EDuration="23.293678105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:38.293386007 +0000 UTC m=+25.552335867" watchObservedRunningTime="2026-07-09 09:54:38.293678105 +0000 UTC m=+25.552627961" Jul 09 09:54:38.930266 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:38.930098 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T09:54:38.083929837Z","UUID":"ee46b2d0-4ce4-484c-8bcc-09d7281b2990","Handler":null,"Name":"","Endpoint":""} Jul 09 09:54:38.931698 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:38.931677 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 09:54:38.931827 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:38.931730 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 09:54:39.034852 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:39.034820 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:39.034852 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:39.034849 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:39.035121 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:39.034949 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:39.035121 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:39.034980 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:39.035121 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:39.035058 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:39.035290 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:39.035128 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:39.282350 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:39.282316 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" event={"ID":"32dd5ea2-f421-45fd-b2c8-5e27535f407c","Type":"ContainerStarted","Data":"fa52bd8d6f4029d35ee9c4ef289ed1c85d1074fc3ea8ef0103b967f486331b7f"} Jul 09 09:54:39.989669 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:39.989623 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v97nr" Jul 09 09:54:39.990289 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:39.990263 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v97nr" Jul 09 09:54:40.286442 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:40.286360 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" event={"ID":"b5ab5674-a77b-456f-a530-e47b0fb0e33c","Type":"ContainerStarted","Data":"8f3a1a2f32c968f1ad6b649d5073d2544f21e54a16474711d2284fd0f48f98ef"} Jul 09 09:54:40.286880 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:40.286571 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v97nr" Jul 09 09:54:40.287119 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:40.287101 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v97nr" Jul 09 09:54:40.306115 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:40.306072 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-s5t8g" podStartSLOduration=2.601484453 podStartE2EDuration="26.30605878s" totalImagesPullingTime="23.704574327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:40.305577536 +0000 UTC m=+27.564527397" watchObservedRunningTime="2026-07-09 09:54:40.30605878 +0000 UTC m=+27.565008638" Jul 09 09:54:41.035080 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:41.035042 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:41.035281 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:41.035093 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:41.035281 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:41.035093 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:41.035281 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:41.035170 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:41.035281 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:41.035269 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:41.035476 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:41.035363 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:41.292537 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:41.292500 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" event={"ID":"32dd5ea2-f421-45fd-b2c8-5e27535f407c","Type":"ContainerStarted","Data":"02c8a311256334ff142303c0a8bb19e9224aa33d7d0feede7033927821e5ebe8"} Jul 09 09:54:41.329371 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:41.329305 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" podStartSLOduration=6.645499228 podStartE2EDuration="27.329284291s" totalImagesPullingTime="20.683785063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:41.328656713 +0000 UTC m=+28.587606573" watchObservedRunningTime="2026-07-09 09:54:41.329284291 +0000 UTC m=+28.588234152" Jul 09 09:54:42.295977 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:42.295943 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:42.296423 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:42.296216 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:42.296423 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:42.296291 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:42.313902 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:42.313874 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:42.314253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:42.314229 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:54:43.034683 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:43.034651 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:43.034849 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:43.034692 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:43.034849 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:43.034741 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:43.034849 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:43.034755 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:43.034849 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:43.034828 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:43.034984 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:43.034896 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:43.122694 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:43.122657 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f987x"] Jul 09 09:54:43.125488 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:43.125459 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fg9gs"] Jul 09 09:54:43.126260 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:43.126241 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7fx9w"] Jul 09 09:54:43.302773 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:43.302741 2582 generic.go:356] "Generic (PLEG): container finished" podID="af327940-c861-4625-a6bb-1fbfdf1d0cb2" containerID="fdf5445131b719250ae5747bc271448aa80ee29f01bca2ad4dfeaef1284b1707" exitCode=0 Jul 09 09:54:43.303359 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:43.302816 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:43.303359 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:43.302827 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nktqq" event={"ID":"af327940-c861-4625-a6bb-1fbfdf1d0cb2","Type":"ContainerDied","Data":"fdf5445131b719250ae5747bc271448aa80ee29f01bca2ad4dfeaef1284b1707"} Jul 09 09:54:43.303359 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:43.302862 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:43.303359 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:43.303042 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:43.303359 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:43.303072 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:43.303359 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:43.303125 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:43.303359 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:43.303206 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:44.307050 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:44.307021 2582 generic.go:356] "Generic (PLEG): container finished" podID="af327940-c861-4625-a6bb-1fbfdf1d0cb2" containerID="6745c3bb384b4a496d72c1d1d06ffb8b1b55fd55e5d08d30c3d9cf91a50cebed" exitCode=0 Jul 09 09:54:44.307504 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:44.307098 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nktqq" event={"ID":"af327940-c861-4625-a6bb-1fbfdf1d0cb2","Type":"ContainerDied","Data":"6745c3bb384b4a496d72c1d1d06ffb8b1b55fd55e5d08d30c3d9cf91a50cebed"} Jul 09 09:54:45.034662 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:45.034636 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:45.034740 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:45.034667 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:45.034740 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:45.034667 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:45.034808 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:45.034762 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:45.034843 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:45.034815 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:45.034896 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:45.034879 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:45.311783 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:45.311746 2582 generic.go:356] "Generic (PLEG): container finished" podID="af327940-c861-4625-a6bb-1fbfdf1d0cb2" containerID="6d0875d10348c1f64d594dc8bef456bf2f1b53eab88c3bbfbabd3bb6e686ad20" exitCode=0 Jul 09 09:54:45.312211 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:45.311800 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nktqq" event={"ID":"af327940-c861-4625-a6bb-1fbfdf1d0cb2","Type":"ContainerDied","Data":"6d0875d10348c1f64d594dc8bef456bf2f1b53eab88c3bbfbabd3bb6e686ad20"} Jul 09 09:54:46.619578 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:46.619541 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:46.620231 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:46.619716 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:46.620231 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:46.619775 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs podName:ab3cadc2-2f43-4d65-95a2-d83bae26ec1e nodeName:}" failed. No retries permitted until 2026-07-09 09:55:18.619759364 +0000 UTC m=+65.878709206 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs") pod "network-metrics-daemon-7fx9w" (UID: "ab3cadc2-2f43-4d65-95a2-d83bae26ec1e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 09:54:46.719936 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:46.719905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:46.720098 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:46.720024 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:46.720098 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:46.720075 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret podName:e0bd29c0-1d95-4097-9628-94da4639bcd5 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:18.720061724 +0000 UTC m=+65.979011561 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret") pod "global-pull-secret-syncer-f987x" (UID: "e0bd29c0-1d95-4097-9628-94da4639bcd5") : object "kube-system"/"original-pull-secret" not registered Jul 09 09:54:46.820736 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:46.820701 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57g88\" (UniqueName: \"kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88\") pod \"network-check-target-fg9gs\" (UID: \"42a23145-ce54-4f13-856c-b112538017d0\") " pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:46.820899 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:46.820881 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 09:54:46.820962 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:46.820906 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 09:54:46.820962 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:46.820920 2582 projected.go:196] Error preparing data for projected volume kube-api-access-57g88 for pod openshift-network-diagnostics/network-check-target-fg9gs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:46.821079 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:46.820970 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88 podName:42a23145-ce54-4f13-856c-b112538017d0 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:18.82095644 +0000 UTC m=+66.079906281 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-57g88" (UniqueName: "kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88") pod "network-check-target-fg9gs" (UID: "42a23145-ce54-4f13-856c-b112538017d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 09:54:47.034586 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:47.034558 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:47.034735 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:47.034674 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:47.034735 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:47.034683 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f987x" podUID="e0bd29c0-1d95-4097-9628-94da4639bcd5" Jul 09 09:54:47.034813 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:47.034760 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fg9gs" podUID="42a23145-ce54-4f13-856c-b112538017d0" Jul 09 09:54:47.034813 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:47.034785 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:47.034867 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:47.034847 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7fx9w" podUID="ab3cadc2-2f43-4d65-95a2-d83bae26ec1e" Jul 09 09:54:48.330506 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.330323 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-chwp7_8edce7b1-4310-4adf-b754-3169b53b9a90/dns-node-resolver/0.log" Jul 09 09:54:48.547493 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.547467 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-220.ec2.internal" event="NodeReady" Jul 09 09:54:48.547674 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.547594 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 09:54:48.582389 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.582307 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-99b7b67cd-m9x6s"] Jul 09 09:54:48.612905 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.612876 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n2dcz"] Jul 09 09:54:48.613076 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.612972 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.615577 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.615552 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 09:54:48.615710 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.615684 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 09:54:48.615710 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.615692 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 09:54:48.615814 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.615777 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jpdnh\"" Jul 09 09:54:48.621696 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.621672 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 09:54:48.627725 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.627704 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6znj8"] Jul 09 09:54:48.627846 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.627831 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n2dcz" Jul 09 09:54:48.630807 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.630785 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 09:54:48.630920 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.630821 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-df4wp\"" Jul 09 09:54:48.630972 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.630943 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 09:54:48.631057 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.631040 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 09:54:48.643869 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.643851 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-99b7b67cd-m9x6s"] Jul 09 09:54:48.643967 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.643881 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n2dcz"] Jul 09 09:54:48.643967 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.643895 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6znj8"] Jul 09 09:54:48.643967 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.643924 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ngzg4"] Jul 09 09:54:48.644090 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.644052 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:48.646840 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.646632 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 09:54:48.646840 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.646688 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 09:54:48.646840 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.646713 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n6pb2\"" Jul 09 09:54:48.664339 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.664320 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ngzg4"] Jul 09 09:54:48.664429 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.664399 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.666781 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.666764 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 09:54:48.667899 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.667483 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 09:54:48.667899 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.667500 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zkpxd\"" Jul 09 09:54:48.667899 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.667519 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 09:54:48.667899 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.667760 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 09:54:48.736203 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736169 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-tmp-dir\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:48.736359 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736215 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.736359 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736294 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a502f840-af4d-417d-9113-35feb0400984-cert\") pod \"ingress-canary-n2dcz\" (UID: \"a502f840-af4d-417d-9113-35feb0400984\") " pod="openshift-ingress-canary/ingress-canary-n2dcz" Jul 09 09:54:48.736476 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736376 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-bound-sa-token\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.736476 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736422 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9sh2\" (UniqueName: \"kubernetes.io/projected/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-kube-api-access-j9sh2\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:48.736553 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736475 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/752bfa27-9e1f-438c-a490-70458fd60715-ca-trust-extracted\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.736553 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736492 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/752bfa27-9e1f-438c-a490-70458fd60715-trusted-ca\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.736553 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736513 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:48.736714 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736607 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/752bfa27-9e1f-438c-a490-70458fd60715-image-registry-private-configuration\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.736714 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736654 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzcvr\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-kube-api-access-jzcvr\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.736714 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736683 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj7hp\" (UniqueName: \"kubernetes.io/projected/a502f840-af4d-417d-9113-35feb0400984-kube-api-access-vj7hp\") pod \"ingress-canary-n2dcz\" (UID: \"a502f840-af4d-417d-9113-35feb0400984\") " pod="openshift-ingress-canary/ingress-canary-n2dcz" Jul 09 09:54:48.736843 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736724 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/752bfa27-9e1f-438c-a490-70458fd60715-registry-certificates\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.736843 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736769 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/752bfa27-9e1f-438c-a490-70458fd60715-installation-pull-secrets\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.736843 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.736808 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-config-volume\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:48.837964 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.837891 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a502f840-af4d-417d-9113-35feb0400984-cert\") pod \"ingress-canary-n2dcz\" (UID: \"a502f840-af4d-417d-9113-35feb0400984\") " pod="openshift-ingress-canary/ingress-canary-n2dcz" Jul 09 09:54:48.838106 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.837950 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-bound-sa-token\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.838106 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838003 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9sh2\" (UniqueName: \"kubernetes.io/projected/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-kube-api-access-j9sh2\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:48.838106 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838026 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/752bfa27-9e1f-438c-a490-70458fd60715-ca-trust-extracted\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.838250 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838182 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/752bfa27-9e1f-438c-a490-70458fd60715-trusted-ca\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.838459 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838444 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:48.838494 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838472 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.838530 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838509 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/752bfa27-9e1f-438c-a490-70458fd60715-image-registry-private-configuration\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.838530 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838526 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzcvr\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-kube-api-access-jzcvr\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.838601 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838541 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj7hp\" (UniqueName: \"kubernetes.io/projected/a502f840-af4d-417d-9113-35feb0400984-kube-api-access-vj7hp\") pod \"ingress-canary-n2dcz\" (UID: \"a502f840-af4d-417d-9113-35feb0400984\") " pod="openshift-ingress-canary/ingress-canary-n2dcz" Jul 09 09:54:48.838601 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838550 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/752bfa27-9e1f-438c-a490-70458fd60715-ca-trust-extracted\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.838601 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838561 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/752bfa27-9e1f-438c-a490-70458fd60715-registry-certificates\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.838714 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838646 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/752bfa27-9e1f-438c-a490-70458fd60715-installation-pull-secrets\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.838714 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838674 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-config-volume\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:48.838714 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838701 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8b9d4efd-4145-49fd-a355-042b72301086-crio-socket\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.838827 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838728 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgnj7\" (UniqueName: \"kubernetes.io/projected/8b9d4efd-4145-49fd-a355-042b72301086-kube-api-access-lgnj7\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.838827 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838770 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b9d4efd-4145-49fd-a355-042b72301086-data-volume\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.838827 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838798 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-tmp-dir\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:48.838967 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838836 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.838967 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.838859 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.839060 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.839042 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/752bfa27-9e1f-438c-a490-70458fd60715-registry-certificates\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.839159 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.839135 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/752bfa27-9e1f-438c-a490-70458fd60715-trusted-ca\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.839225 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:48.839145 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 09:54:48.839225 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.839210 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-tmp-dir\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:48.839225 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:48.839223 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls podName:d1a11a4f-b946-4d7b-8540-d56dc533c8f0 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:49.339203346 +0000 UTC m=+36.598153183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls") pod "dns-default-6znj8" (UID: "d1a11a4f-b946-4d7b-8540-d56dc533c8f0") : secret "dns-default-metrics-tls" not found Jul 09 09:54:48.839225 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:48.839222 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 09:54:48.839401 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:48.839238 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-99b7b67cd-m9x6s: secret "image-registry-tls" not found Jul 09 09:54:48.839401 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:48.839265 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls podName:752bfa27-9e1f-438c-a490-70458fd60715 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:49.339257175 +0000 UTC m=+36.598207041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls") pod "image-registry-99b7b67cd-m9x6s" (UID: "752bfa27-9e1f-438c-a490-70458fd60715") : secret "image-registry-tls" not found Jul 09 09:54:48.839804 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.839776 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-config-volume\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:48.843012 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.842983 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/752bfa27-9e1f-438c-a490-70458fd60715-image-registry-private-configuration\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.843138 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.842978 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/752bfa27-9e1f-438c-a490-70458fd60715-installation-pull-secrets\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.843138 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.842993 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a502f840-af4d-417d-9113-35feb0400984-cert\") pod \"ingress-canary-n2dcz\" (UID: \"a502f840-af4d-417d-9113-35feb0400984\") " pod="openshift-ingress-canary/ingress-canary-n2dcz" Jul 09 09:54:48.848411 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.848384 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9sh2\" (UniqueName: \"kubernetes.io/projected/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-kube-api-access-j9sh2\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:48.848554 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.848455 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj7hp\" (UniqueName: \"kubernetes.io/projected/a502f840-af4d-417d-9113-35feb0400984-kube-api-access-vj7hp\") pod \"ingress-canary-n2dcz\" (UID: \"a502f840-af4d-417d-9113-35feb0400984\") " pod="openshift-ingress-canary/ingress-canary-n2dcz" Jul 09 09:54:48.849200 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.849180 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzcvr\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-kube-api-access-jzcvr\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.849289 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.849185 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-bound-sa-token\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:48.938486 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.938455 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n2dcz" Jul 09 09:54:48.939810 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.939786 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.939887 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.939867 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8b9d4efd-4145-49fd-a355-042b72301086-crio-socket\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.939948 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.939892 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgnj7\" (UniqueName: \"kubernetes.io/projected/8b9d4efd-4145-49fd-a355-042b72301086-kube-api-access-lgnj7\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.939948 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.939912 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b9d4efd-4145-49fd-a355-042b72301086-data-volume\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.939948 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:48.939920 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 09:54:48.939948 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.939942 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.940148 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:48.940007 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls podName:8b9d4efd-4145-49fd-a355-042b72301086 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:49.439983351 +0000 UTC m=+36.698933192 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ngzg4" (UID: "8b9d4efd-4145-49fd-a355-042b72301086") : secret "insights-runtime-extractor-tls" not found Jul 09 09:54:48.940148 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:48.940010 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 09:54:48.940148 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:48.940059 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm podName:8b9d4efd-4145-49fd-a355-042b72301086 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:49.440048192 +0000 UTC m=+36.698998052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ngzg4" (UID: "8b9d4efd-4145-49fd-a355-042b72301086") : configmap "kube-rbac-proxy" not found Jul 09 09:54:48.940148 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.940124 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8b9d4efd-4145-49fd-a355-042b72301086-crio-socket\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.940380 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.940350 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8b9d4efd-4145-49fd-a355-042b72301086-data-volume\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:48.950918 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:48.950897 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgnj7\" (UniqueName: \"kubernetes.io/projected/8b9d4efd-4145-49fd-a355-042b72301086-kube-api-access-lgnj7\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:49.035509 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.035476 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:54:49.035703 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.035602 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:54:49.035703 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.035643 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:54:49.045238 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.045173 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 09:54:49.045238 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.045234 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4794j\"" Jul 09 09:54:49.045421 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.045285 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 09:54:49.045421 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.045174 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 09:54:49.045699 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.045680 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 09:54:49.045784 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.045683 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ffmqr\"" Jul 09 09:54:49.342536 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.342501 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:49.343026 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.342601 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:49.343026 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:49.342661 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 09:54:49.343026 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:49.342704 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 09:54:49.343026 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:49.342717 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-99b7b67cd-m9x6s: secret "image-registry-tls" not found Jul 09 09:54:49.343026 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:49.342747 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls podName:d1a11a4f-b946-4d7b-8540-d56dc533c8f0 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:50.342724619 +0000 UTC m=+37.601674457 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls") pod "dns-default-6znj8" (UID: "d1a11a4f-b946-4d7b-8540-d56dc533c8f0") : secret "dns-default-metrics-tls" not found Jul 09 09:54:49.343026 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:49.342766 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls podName:752bfa27-9e1f-438c-a490-70458fd60715 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:50.342757629 +0000 UTC m=+37.601707468 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls") pod "image-registry-99b7b67cd-m9x6s" (UID: "752bfa27-9e1f-438c-a490-70458fd60715") : secret "image-registry-tls" not found Jul 09 09:54:49.443529 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.443489 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:49.443745 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.443587 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:49.443745 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:49.443665 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 09:54:49.443745 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:49.443723 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 09:54:49.443912 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:49.443767 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls podName:8b9d4efd-4145-49fd-a355-042b72301086 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:50.443750919 +0000 UTC m=+37.702700760 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ngzg4" (UID: "8b9d4efd-4145-49fd-a355-042b72301086") : secret "insights-runtime-extractor-tls" not found Jul 09 09:54:49.443912 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:49.443786 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm podName:8b9d4efd-4145-49fd-a355-042b72301086 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:50.443777109 +0000 UTC m=+37.702726951 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ngzg4" (UID: "8b9d4efd-4145-49fd-a355-042b72301086") : configmap "kube-rbac-proxy" not found Jul 09 09:54:49.507036 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.507014 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-grjk4_8017af28-6540-46c3-9bc3-8b4ab3b83303/node-ca/0.log" Jul 09 09:54:49.810448 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:49.810419 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n2dcz"] Jul 09 09:54:49.873795 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:49.873757 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda502f840_af4d_417d_9113_35feb0400984.slice/crio-8af73e2c047cc6644a527207cb7587fd8d3e1462495b258cc230e80a3f3348db WatchSource:0}: Error finding container 8af73e2c047cc6644a527207cb7587fd8d3e1462495b258cc230e80a3f3348db: Status 404 returned error can't find the container with id 8af73e2c047cc6644a527207cb7587fd8d3e1462495b258cc230e80a3f3348db Jul 09 09:54:50.326923 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:50.326573 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n2dcz" event={"ID":"a502f840-af4d-417d-9113-35feb0400984","Type":"ContainerStarted","Data":"8af73e2c047cc6644a527207cb7587fd8d3e1462495b258cc230e80a3f3348db"} Jul 09 09:54:50.329996 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:50.329967 2582 generic.go:356] "Generic (PLEG): container finished" podID="af327940-c861-4625-a6bb-1fbfdf1d0cb2" containerID="0e1c82b06f88293ef0c20bfcf09c954dd9f72b43115608cb3135c3b5a6a252bd" exitCode=0 Jul 09 09:54:50.330128 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:50.330014 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nktqq" event={"ID":"af327940-c861-4625-a6bb-1fbfdf1d0cb2","Type":"ContainerDied","Data":"0e1c82b06f88293ef0c20bfcf09c954dd9f72b43115608cb3135c3b5a6a252bd"} Jul 09 09:54:50.350234 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:50.350208 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:50.350955 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:50.350280 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:50.350955 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:50.350367 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 09:54:50.350955 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:50.350387 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-99b7b67cd-m9x6s: secret "image-registry-tls" not found Jul 09 09:54:50.350955 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:50.350421 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 09:54:50.350955 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:50.350447 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls podName:752bfa27-9e1f-438c-a490-70458fd60715 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:52.350427573 +0000 UTC m=+39.609377413 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls") pod "image-registry-99b7b67cd-m9x6s" (UID: "752bfa27-9e1f-438c-a490-70458fd60715") : secret "image-registry-tls" not found Jul 09 09:54:50.350955 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:50.350464 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls podName:d1a11a4f-b946-4d7b-8540-d56dc533c8f0 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:52.350453241 +0000 UTC m=+39.609403078 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls") pod "dns-default-6znj8" (UID: "d1a11a4f-b946-4d7b-8540-d56dc533c8f0") : secret "dns-default-metrics-tls" not found Jul 09 09:54:50.451021 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:50.450989 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:50.451160 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:50.451115 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:50.451160 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:50.451120 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 09:54:50.451271 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:50.451191 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm podName:8b9d4efd-4145-49fd-a355-042b72301086 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:52.45117556 +0000 UTC m=+39.710125400 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ngzg4" (UID: "8b9d4efd-4145-49fd-a355-042b72301086") : configmap "kube-rbac-proxy" not found Jul 09 09:54:50.451271 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:50.451259 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 09:54:50.451401 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:50.451313 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls podName:8b9d4efd-4145-49fd-a355-042b72301086 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:52.451293929 +0000 UTC m=+39.710243770 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ngzg4" (UID: "8b9d4efd-4145-49fd-a355-042b72301086") : secret "insights-runtime-extractor-tls" not found Jul 09 09:54:51.336333 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:51.336299 2582 generic.go:356] "Generic (PLEG): container finished" podID="af327940-c861-4625-a6bb-1fbfdf1d0cb2" containerID="50989dc184bcb2eee60664e6cc9440dabd412ce1a4e2f4e28d06b8a74dc30a67" exitCode=0 Jul 09 09:54:51.336527 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:51.336351 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nktqq" event={"ID":"af327940-c861-4625-a6bb-1fbfdf1d0cb2","Type":"ContainerDied","Data":"50989dc184bcb2eee60664e6cc9440dabd412ce1a4e2f4e28d06b8a74dc30a67"} Jul 09 09:54:52.342204 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:52.342176 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nktqq" event={"ID":"af327940-c861-4625-a6bb-1fbfdf1d0cb2","Type":"ContainerStarted","Data":"02bdaeaef501637495504656b89e640d427a536cafcb4b7298492d73e2afe27b"} Jul 09 09:54:52.365389 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:52.365367 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:52.365472 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:52.365431 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:52.365547 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:52.365522 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 09:54:52.365547 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:52.365542 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-99b7b67cd-m9x6s: secret "image-registry-tls" not found Jul 09 09:54:52.365547 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:52.365548 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 09:54:52.365747 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:52.365595 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls podName:d1a11a4f-b946-4d7b-8540-d56dc533c8f0 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:56.36557893 +0000 UTC m=+43.624528771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls") pod "dns-default-6znj8" (UID: "d1a11a4f-b946-4d7b-8540-d56dc533c8f0") : secret "dns-default-metrics-tls" not found Jul 09 09:54:52.365747 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:52.365624 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls podName:752bfa27-9e1f-438c-a490-70458fd60715 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:56.365603218 +0000 UTC m=+43.624553056 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls") pod "image-registry-99b7b67cd-m9x6s" (UID: "752bfa27-9e1f-438c-a490-70458fd60715") : secret "image-registry-tls" not found Jul 09 09:54:52.382530 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:52.382481 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nktqq" podStartSLOduration=6.595232469 podStartE2EDuration="38.382461672s" totalImagesPullingTime="31.787229203s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:14 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:52.380779622 +0000 UTC m=+39.639729480" watchObservedRunningTime="2026-07-09 09:54:52.382461672 +0000 UTC m=+39.641411579" Jul 09 09:54:52.466321 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:52.466289 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:52.466462 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:52.466378 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:52.466462 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:52.466423 2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 09:54:52.466585 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:52.466473 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 09:54:52.466585 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:52.466481 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm podName:8b9d4efd-4145-49fd-a355-042b72301086 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:56.466463954 +0000 UTC m=+43.725413814 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ngzg4" (UID: "8b9d4efd-4145-49fd-a355-042b72301086") : configmap "kube-rbac-proxy" not found Jul 09 09:54:52.466585 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:52.466512 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls podName:8b9d4efd-4145-49fd-a355-042b72301086 nodeName:}" failed. No retries permitted until 2026-07-09 09:54:56.466505637 +0000 UTC m=+43.725455475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ngzg4" (UID: "8b9d4efd-4145-49fd-a355-042b72301086") : secret "insights-runtime-extractor-tls" not found Jul 09 09:54:53.346473 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:53.346437 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n2dcz" event={"ID":"a502f840-af4d-417d-9113-35feb0400984","Type":"ContainerStarted","Data":"b5ae360d9296a394125e99d7417063da716ae7330c173eaf0aa5fc70d46f4a74"} Jul 09 09:54:53.364523 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:53.364468 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n2dcz" podStartSLOduration=2.916537937 podStartE2EDuration="5.36445307s" totalImagesPullingTime="2.447915133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:53.364245551 +0000 UTC m=+40.623195400" watchObservedRunningTime="2026-07-09 09:54:53.36445307 +0000 UTC m=+40.623402962" Jul 09 09:54:53.985860 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:53.985829 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-wnjcz"] Jul 09 09:54:54.017582 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:54.017553 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-wnjcz"] Jul 09 09:54:54.017750 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:54.017688 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-wnjcz" Jul 09 09:54:54.026823 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:54.026795 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 09 09:54:54.027940 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:54.027922 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 09 09:54:54.039807 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:54.039787 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-svldg\"" Jul 09 09:54:54.178232 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:54.178191 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gs62\" (UniqueName: \"kubernetes.io/projected/8f8e95c4-cc5e-4a98-9710-6ed6d26a24d4-kube-api-access-4gs62\") pod \"migrator-d57d58cc9-wnjcz\" (UID: \"8f8e95c4-cc5e-4a98-9710-6ed6d26a24d4\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-wnjcz" Jul 09 09:54:54.279298 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:54.279209 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gs62\" (UniqueName: \"kubernetes.io/projected/8f8e95c4-cc5e-4a98-9710-6ed6d26a24d4-kube-api-access-4gs62\") pod \"migrator-d57d58cc9-wnjcz\" (UID: \"8f8e95c4-cc5e-4a98-9710-6ed6d26a24d4\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-wnjcz" Jul 09 09:54:54.296114 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:54.296087 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gs62\" (UniqueName: \"kubernetes.io/projected/8f8e95c4-cc5e-4a98-9710-6ed6d26a24d4-kube-api-access-4gs62\") pod \"migrator-d57d58cc9-wnjcz\" (UID: \"8f8e95c4-cc5e-4a98-9710-6ed6d26a24d4\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-wnjcz" Jul 09 09:54:54.328076 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:54.328052 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-wnjcz" Jul 09 09:54:54.465918 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:54.465895 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-wnjcz"] Jul 09 09:54:54.468083 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:54.468056 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8e95c4_cc5e_4a98_9710_6ed6d26a24d4.slice/crio-16c0beda32ef7d20aea1195b567c81241f8d5dfaf9df762d99c3434d06ec7ea5 WatchSource:0}: Error finding container 16c0beda32ef7d20aea1195b567c81241f8d5dfaf9df762d99c3434d06ec7ea5: Status 404 returned error can't find the container with id 16c0beda32ef7d20aea1195b567c81241f8d5dfaf9df762d99c3434d06ec7ea5 Jul 09 09:54:55.352079 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:55.352038 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-wnjcz" event={"ID":"8f8e95c4-cc5e-4a98-9710-6ed6d26a24d4","Type":"ContainerStarted","Data":"16c0beda32ef7d20aea1195b567c81241f8d5dfaf9df762d99c3434d06ec7ea5"} Jul 09 09:54:55.842425 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:55.842358 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-8gbl8"] Jul 09 09:54:55.874739 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:55.874705 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-8gbl8"] Jul 09 09:54:55.874868 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:55.874791 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-8gbl8" Jul 09 09:54:55.878201 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:55.878178 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 09 09:54:55.879485 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:55.879463 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 09 09:54:55.879485 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:55.879459 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-2r5wf\"" Jul 09 09:54:55.879628 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:55.879527 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 09 09:54:55.879628 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:55.879465 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 09 09:54:55.992038 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:55.992008 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bba90984-871e-437c-b783-65e1a23a9052-signing-cabundle\") pod \"service-ca-758f599c5-8gbl8\" (UID: \"bba90984-871e-437c-b783-65e1a23a9052\") " pod="openshift-service-ca/service-ca-758f599c5-8gbl8" Jul 09 09:54:55.992182 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:55.992095 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bba90984-871e-437c-b783-65e1a23a9052-signing-key\") pod \"service-ca-758f599c5-8gbl8\" (UID: \"bba90984-871e-437c-b783-65e1a23a9052\") " pod="openshift-service-ca/service-ca-758f599c5-8gbl8" Jul 09 09:54:55.992182 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:55.992112 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzvtv\" (UniqueName: \"kubernetes.io/projected/bba90984-871e-437c-b783-65e1a23a9052-kube-api-access-tzvtv\") pod \"service-ca-758f599c5-8gbl8\" (UID: \"bba90984-871e-437c-b783-65e1a23a9052\") " pod="openshift-service-ca/service-ca-758f599c5-8gbl8" Jul 09 09:54:56.093064 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.092979 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bba90984-871e-437c-b783-65e1a23a9052-signing-key\") pod \"service-ca-758f599c5-8gbl8\" (UID: \"bba90984-871e-437c-b783-65e1a23a9052\") " pod="openshift-service-ca/service-ca-758f599c5-8gbl8" Jul 09 09:54:56.093064 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.093016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzvtv\" (UniqueName: \"kubernetes.io/projected/bba90984-871e-437c-b783-65e1a23a9052-kube-api-access-tzvtv\") pod \"service-ca-758f599c5-8gbl8\" (UID: \"bba90984-871e-437c-b783-65e1a23a9052\") " pod="openshift-service-ca/service-ca-758f599c5-8gbl8" Jul 09 09:54:56.093064 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.093062 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bba90984-871e-437c-b783-65e1a23a9052-signing-cabundle\") pod \"service-ca-758f599c5-8gbl8\" (UID: \"bba90984-871e-437c-b783-65e1a23a9052\") " pod="openshift-service-ca/service-ca-758f599c5-8gbl8" Jul 09 09:54:56.093763 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.093741 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/bba90984-871e-437c-b783-65e1a23a9052-signing-cabundle\") pod \"service-ca-758f599c5-8gbl8\" (UID: \"bba90984-871e-437c-b783-65e1a23a9052\") " pod="openshift-service-ca/service-ca-758f599c5-8gbl8" Jul 09 09:54:56.095844 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.095818 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/bba90984-871e-437c-b783-65e1a23a9052-signing-key\") pod \"service-ca-758f599c5-8gbl8\" (UID: \"bba90984-871e-437c-b783-65e1a23a9052\") " pod="openshift-service-ca/service-ca-758f599c5-8gbl8" Jul 09 09:54:56.103430 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.103408 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzvtv\" (UniqueName: \"kubernetes.io/projected/bba90984-871e-437c-b783-65e1a23a9052-kube-api-access-tzvtv\") pod \"service-ca-758f599c5-8gbl8\" (UID: \"bba90984-871e-437c-b783-65e1a23a9052\") " pod="openshift-service-ca/service-ca-758f599c5-8gbl8" Jul 09 09:54:56.184144 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.184112 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-8gbl8" Jul 09 09:54:56.331206 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.331169 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-8gbl8"] Jul 09 09:54:56.334061 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:54:56.334032 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba90984_871e_437c_b783_65e1a23a9052.slice/crio-89cbf1331a4fc1b6a9e9a8fd7820e0c4ff0911cbac77352cb0c077bb4c4bfd8d WatchSource:0}: Error finding container 89cbf1331a4fc1b6a9e9a8fd7820e0c4ff0911cbac77352cb0c077bb4c4bfd8d: Status 404 returned error can't find the container with id 89cbf1331a4fc1b6a9e9a8fd7820e0c4ff0911cbac77352cb0c077bb4c4bfd8d Jul 09 09:54:56.354785 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.354727 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-8gbl8" event={"ID":"bba90984-871e-437c-b783-65e1a23a9052","Type":"ContainerStarted","Data":"89cbf1331a4fc1b6a9e9a8fd7820e0c4ff0911cbac77352cb0c077bb4c4bfd8d"} Jul 09 09:54:56.395202 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.395171 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:54:56.395362 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.395236 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:54:56.395443 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:56.395375 2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 09:54:56.395443 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:56.395403 2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-99b7b67cd-m9x6s: secret "image-registry-tls" not found Jul 09 09:54:56.395443 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:56.395372 2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 09:54:56.395659 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:56.395482 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls podName:752bfa27-9e1f-438c-a490-70458fd60715 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:04.395460141 +0000 UTC m=+51.654409980 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls") pod "image-registry-99b7b67cd-m9x6s" (UID: "752bfa27-9e1f-438c-a490-70458fd60715") : secret "image-registry-tls" not found Jul 09 09:54:56.395659 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:56.395504 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls podName:d1a11a4f-b946-4d7b-8540-d56dc533c8f0 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:04.395494922 +0000 UTC m=+51.654444762 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls") pod "dns-default-6znj8" (UID: "d1a11a4f-b946-4d7b-8540-d56dc533c8f0") : secret "dns-default-metrics-tls" not found Jul 09 09:54:56.496099 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.496063 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:56.496281 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.496178 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:56.496281 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:56.496235 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 09:54:56.496371 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:54:56.496309 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls podName:8b9d4efd-4145-49fd-a355-042b72301086 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:04.4962882 +0000 UTC m=+51.755238049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ngzg4" (UID: "8b9d4efd-4145-49fd-a355-042b72301086") : secret "insights-runtime-extractor-tls" not found Jul 09 09:54:56.496715 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:56.496693 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8b9d4efd-4145-49fd-a355-042b72301086-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:54:57.362863 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:57.362769 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-wnjcz" event={"ID":"8f8e95c4-cc5e-4a98-9710-6ed6d26a24d4","Type":"ContainerStarted","Data":"a5366e126f36a71ad5990b212f5c84e60cabcf8d91d07cbe8abdda7bba05d253"} Jul 09 09:54:57.363314 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:57.362864 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-wnjcz" event={"ID":"8f8e95c4-cc5e-4a98-9710-6ed6d26a24d4","Type":"ContainerStarted","Data":"144168c8015018b7a381568c38bd063065f0e2056bc33e0d6218437345c463f6"} Jul 09 09:54:57.382629 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:57.382461 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-wnjcz" podStartSLOduration=1.824617658 podStartE2EDuration="4.382448021s" totalImagesPullingTime="2.557830363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:57.38200807 +0000 UTC m=+44.640957932" watchObservedRunningTime="2026-07-09 09:54:57.382448021 +0000 UTC m=+44.641397879" Jul 09 09:54:59.369022 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:59.368945 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-8gbl8" event={"ID":"bba90984-871e-437c-b783-65e1a23a9052","Type":"ContainerStarted","Data":"75a70a878373df82360ed635fe16af32267000973dd5d73f9a625e1eff61e5f7"} Jul 09 09:54:59.387028 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:54:59.386982 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-8gbl8" podStartSLOduration=1.694542154 podStartE2EDuration="4.386969261s" totalImagesPullingTime="2.692427107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:54:59.386180742 +0000 UTC m=+46.645130623" watchObservedRunningTime="2026-07-09 09:54:59.386969261 +0000 UTC m=+46.645919098" Jul 09 09:55:04.460076 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:04.460033 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:55:04.460491 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:04.460149 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:55:04.477477 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:04.477446 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls\") pod \"image-registry-99b7b67cd-m9x6s\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:55:04.489909 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:04.489883 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d1a11a4f-b946-4d7b-8540-d56dc533c8f0-metrics-tls\") pod \"dns-default-6znj8\" (UID: \"d1a11a4f-b946-4d7b-8540-d56dc533c8f0\") " pod="openshift-dns/dns-default-6znj8" Jul 09 09:55:04.525699 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:04.525670 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:55:04.554454 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:04.554419 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6znj8" Jul 09 09:55:04.560896 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:04.560867 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:55:04.561037 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:55:04.560978 2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 09:55:04.561037 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:55:04.561032 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls podName:8b9d4efd-4145-49fd-a355-042b72301086 nodeName:}" failed. No retries permitted until 2026-07-09 09:55:20.561018063 +0000 UTC m=+67.819967904 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ngzg4" (UID: "8b9d4efd-4145-49fd-a355-042b72301086") : secret "insights-runtime-extractor-tls" not found Jul 09 09:55:04.684496 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:04.684448 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-99b7b67cd-m9x6s"] Jul 09 09:55:04.720399 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:04.720376 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6znj8"] Jul 09 09:55:04.728533 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:55:04.728510 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1a11a4f_b946_4d7b_8540_d56dc533c8f0.slice/crio-7339d4daff58768429869c8b574df8054e74e8cf1786b8a6bc6e81ad5e372f0d WatchSource:0}: Error finding container 7339d4daff58768429869c8b574df8054e74e8cf1786b8a6bc6e81ad5e372f0d: Status 404 returned error can't find the container with id 7339d4daff58768429869c8b574df8054e74e8cf1786b8a6bc6e81ad5e372f0d Jul 09 09:55:05.384430 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:05.384397 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6znj8" event={"ID":"d1a11a4f-b946-4d7b-8540-d56dc533c8f0","Type":"ContainerStarted","Data":"7339d4daff58768429869c8b574df8054e74e8cf1786b8a6bc6e81ad5e372f0d"} Jul 09 09:55:05.385875 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:05.385852 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" event={"ID":"752bfa27-9e1f-438c-a490-70458fd60715","Type":"ContainerStarted","Data":"dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881"} Jul 09 09:55:05.386003 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:05.385893 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" event={"ID":"752bfa27-9e1f-438c-a490-70458fd60715","Type":"ContainerStarted","Data":"2d56a5b9b38cdb18f859fc6397f806c2dbb8deb36e44a13fd76c02c12b0f9ae9"} Jul 09 09:55:05.386003 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:05.385972 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:55:05.410855 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:05.410804 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" podStartSLOduration=19.410788542 podStartE2EDuration="19.410788542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:05.410107613 +0000 UTC m=+52.669057473" watchObservedRunningTime="2026-07-09 09:55:05.410788542 +0000 UTC m=+52.669738401" Jul 09 09:55:07.394540 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:07.394505 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6znj8" event={"ID":"d1a11a4f-b946-4d7b-8540-d56dc533c8f0","Type":"ContainerStarted","Data":"87a1c29e73a94302e41d2dc6b9ab0423fc4281357203be46d2afda1ea64ade73"} Jul 09 09:55:08.400269 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:08.400228 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6znj8" event={"ID":"d1a11a4f-b946-4d7b-8540-d56dc533c8f0","Type":"ContainerStarted","Data":"f136744d24e2572a712748ee13e02529216c53d623e36e1416a5ce3818136403"} Jul 09 09:55:08.400674 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:08.400358 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6znj8" Jul 09 09:55:08.419775 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:08.419730 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6znj8" podStartSLOduration=17.934496037 podStartE2EDuration="20.419714607s" totalImagesPullingTime="2.48521857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:08.41912101 +0000 UTC m=+55.678070870" watchObservedRunningTime="2026-07-09 09:55:08.419714607 +0000 UTC m=+55.678664466" Jul 09 09:55:14.322339 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:14.322309 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-psrgx" Jul 09 09:55:15.802679 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.802637 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2"] Jul 09 09:55:15.805297 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.805278 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2" Jul 09 09:55:15.809543 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.809515 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-tjb5v\"" Jul 09 09:55:15.810757 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.810737 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 09 09:55:15.810943 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.810772 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 09 09:55:15.811964 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.811934 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l"] Jul 09 09:55:15.813490 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.813473 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l" Jul 09 09:55:15.816496 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.816477 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k"] Jul 09 09:55:15.818074 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.818053 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 09 09:55:15.818300 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.818281 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.818453 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.818432 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 09 09:55:15.818841 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.818821 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 09 09:55:15.818921 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.818881 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 09 09:55:15.819291 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.819149 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7r4kx\"" Jul 09 09:55:15.824865 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.824846 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 09 09:55:15.825506 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.825490 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 09 09:55:15.826213 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.826199 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 09 09:55:15.827056 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.827039 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 09 09:55:15.833266 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.833248 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2"] Jul 09 09:55:15.837018 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.836997 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l"] Jul 09 09:55:15.842001 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.841972 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k"] Jul 09 09:55:15.844317 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.844286 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e49bf34e-61ba-4df7-9daf-4ec806b9b455-ca\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.844417 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.844334 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e49bf34e-61ba-4df7-9daf-4ec806b9b455-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.844489 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.844458 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e49bf34e-61ba-4df7-9daf-4ec806b9b455-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.844572 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.844553 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b9cbbd8d-4aad-4f42-9a60-31c56912a936-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74669d5c94-ltv2l\" (UID: \"b9cbbd8d-4aad-4f42-9a60-31c56912a936\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l" Jul 09 09:55:15.844654 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.844591 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/58cfca84-a233-4ff9-b44f-1142e072a68b-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-6ndl2\" (UID: \"58cfca84-a233-4ff9-b44f-1142e072a68b\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2" Jul 09 09:55:15.844654 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.844646 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgl2p\" (UniqueName: \"kubernetes.io/projected/b9cbbd8d-4aad-4f42-9a60-31c56912a936-kube-api-access-zgl2p\") pod \"managed-serviceaccount-addon-agent-74669d5c94-ltv2l\" (UID: \"b9cbbd8d-4aad-4f42-9a60-31c56912a936\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l" Jul 09 09:55:15.844770 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.844676 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq2wj\" (UniqueName: \"kubernetes.io/projected/e49bf34e-61ba-4df7-9daf-4ec806b9b455-kube-api-access-jq2wj\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.844770 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.844705 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/58cfca84-a233-4ff9-b44f-1142e072a68b-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-6ndl2\" (UID: \"58cfca84-a233-4ff9-b44f-1142e072a68b\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2" Jul 09 09:55:15.844770 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.844731 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e49bf34e-61ba-4df7-9daf-4ec806b9b455-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.844770 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.844755 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e49bf34e-61ba-4df7-9daf-4ec806b9b455-hub\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.867262 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.867229 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-99b7b67cd-m9x6s"] Jul 09 09:55:15.902635 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.902586 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74df7fc877-768cf"] Jul 09 09:55:15.904318 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.904303 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:15.928059 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.928006 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74df7fc877-768cf"] Jul 09 09:55:15.945437 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945404 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e49bf34e-61ba-4df7-9daf-4ec806b9b455-ca\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.945437 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945439 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e49bf34e-61ba-4df7-9daf-4ec806b9b455-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.945716 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945477 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/38e5e107-6837-4790-a6ba-1f9d2c44c725-installation-pull-secrets\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:15.945716 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945497 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/38e5e107-6837-4790-a6ba-1f9d2c44c725-registry-certificates\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:15.945716 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945524 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e49bf34e-61ba-4df7-9daf-4ec806b9b455-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.945716 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945543 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38e5e107-6837-4790-a6ba-1f9d2c44c725-registry-tls\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:15.945716 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945572 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/38e5e107-6837-4790-a6ba-1f9d2c44c725-image-registry-private-configuration\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:15.945973 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945716 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b9cbbd8d-4aad-4f42-9a60-31c56912a936-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74669d5c94-ltv2l\" (UID: \"b9cbbd8d-4aad-4f42-9a60-31c56912a936\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l" Jul 09 09:55:15.945973 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945753 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q6fs\" (UniqueName: \"kubernetes.io/projected/38e5e107-6837-4790-a6ba-1f9d2c44c725-kube-api-access-7q6fs\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:15.945973 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945787 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/58cfca84-a233-4ff9-b44f-1142e072a68b-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-6ndl2\" (UID: \"58cfca84-a233-4ff9-b44f-1142e072a68b\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2" Jul 09 09:55:15.945973 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945817 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/38e5e107-6837-4790-a6ba-1f9d2c44c725-ca-trust-extracted\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:15.945973 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945846 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38e5e107-6837-4790-a6ba-1f9d2c44c725-bound-sa-token\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:15.945973 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945877 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgl2p\" (UniqueName: \"kubernetes.io/projected/b9cbbd8d-4aad-4f42-9a60-31c56912a936-kube-api-access-zgl2p\") pod \"managed-serviceaccount-addon-agent-74669d5c94-ltv2l\" (UID: \"b9cbbd8d-4aad-4f42-9a60-31c56912a936\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l" Jul 09 09:55:15.945973 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945925 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq2wj\" (UniqueName: \"kubernetes.io/projected/e49bf34e-61ba-4df7-9daf-4ec806b9b455-kube-api-access-jq2wj\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.945973 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945956 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/58cfca84-a233-4ff9-b44f-1142e072a68b-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-6ndl2\" (UID: \"58cfca84-a233-4ff9-b44f-1142e072a68b\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2" Jul 09 09:55:15.946447 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.945985 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e49bf34e-61ba-4df7-9daf-4ec806b9b455-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.946447 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.946012 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e49bf34e-61ba-4df7-9daf-4ec806b9b455-hub\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.946447 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.946037 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38e5e107-6837-4790-a6ba-1f9d2c44c725-trusted-ca\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:15.947263 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.946826 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e49bf34e-61ba-4df7-9daf-4ec806b9b455-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.947263 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.946878 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/58cfca84-a233-4ff9-b44f-1142e072a68b-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-6ndl2\" (UID: \"58cfca84-a233-4ff9-b44f-1142e072a68b\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2" Jul 09 09:55:15.948546 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.948522 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e49bf34e-61ba-4df7-9daf-4ec806b9b455-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.948709 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.948689 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e49bf34e-61ba-4df7-9daf-4ec806b9b455-ca\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.948814 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.948789 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e49bf34e-61ba-4df7-9daf-4ec806b9b455-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.948901 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.948882 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b9cbbd8d-4aad-4f42-9a60-31c56912a936-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74669d5c94-ltv2l\" (UID: \"b9cbbd8d-4aad-4f42-9a60-31c56912a936\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l" Jul 09 09:55:15.948946 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.948895 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e49bf34e-61ba-4df7-9daf-4ec806b9b455-hub\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:15.949520 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.949496 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/58cfca84-a233-4ff9-b44f-1142e072a68b-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-6ndl2\" (UID: \"58cfca84-a233-4ff9-b44f-1142e072a68b\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2" Jul 09 09:55:15.965652 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.965625 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgl2p\" (UniqueName: \"kubernetes.io/projected/b9cbbd8d-4aad-4f42-9a60-31c56912a936-kube-api-access-zgl2p\") pod \"managed-serviceaccount-addon-agent-74669d5c94-ltv2l\" (UID: \"b9cbbd8d-4aad-4f42-9a60-31c56912a936\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l" Jul 09 09:55:15.965965 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:15.965948 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq2wj\" (UniqueName: \"kubernetes.io/projected/e49bf34e-61ba-4df7-9daf-4ec806b9b455-kube-api-access-jq2wj\") pod \"cluster-proxy-proxy-agent-7b86d8b489-l7m6k\" (UID: \"e49bf34e-61ba-4df7-9daf-4ec806b9b455\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:16.046783 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.046752 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/38e5e107-6837-4790-a6ba-1f9d2c44c725-installation-pull-secrets\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.046783 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.046789 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/38e5e107-6837-4790-a6ba-1f9d2c44c725-registry-certificates\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.047044 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.046821 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38e5e107-6837-4790-a6ba-1f9d2c44c725-registry-tls\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.047044 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.046838 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/38e5e107-6837-4790-a6ba-1f9d2c44c725-image-registry-private-configuration\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.047044 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.046913 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q6fs\" (UniqueName: \"kubernetes.io/projected/38e5e107-6837-4790-a6ba-1f9d2c44c725-kube-api-access-7q6fs\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.047044 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.046938 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/38e5e107-6837-4790-a6ba-1f9d2c44c725-ca-trust-extracted\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.047044 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.046964 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38e5e107-6837-4790-a6ba-1f9d2c44c725-bound-sa-token\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.047044 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.047025 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38e5e107-6837-4790-a6ba-1f9d2c44c725-trusted-ca\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.047345 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.047325 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/38e5e107-6837-4790-a6ba-1f9d2c44c725-ca-trust-extracted\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.047741 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.047711 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/38e5e107-6837-4790-a6ba-1f9d2c44c725-registry-certificates\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.048148 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.048123 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38e5e107-6837-4790-a6ba-1f9d2c44c725-trusted-ca\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.049432 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.049405 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/38e5e107-6837-4790-a6ba-1f9d2c44c725-installation-pull-secrets\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.049432 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.049427 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38e5e107-6837-4790-a6ba-1f9d2c44c725-registry-tls\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.049603 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.049497 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/38e5e107-6837-4790-a6ba-1f9d2c44c725-image-registry-private-configuration\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.064943 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.064891 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38e5e107-6837-4790-a6ba-1f9d2c44c725-bound-sa-token\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.065044 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.065028 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q6fs\" (UniqueName: \"kubernetes.io/projected/38e5e107-6837-4790-a6ba-1f9d2c44c725-kube-api-access-7q6fs\") pod \"image-registry-74df7fc877-768cf\" (UID: \"38e5e107-6837-4790-a6ba-1f9d2c44c725\") " pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.113825 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.113791 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2" Jul 09 09:55:16.138319 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.138291 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l" Jul 09 09:55:16.147316 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.147290 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" Jul 09 09:55:16.220310 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.219876 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:16.259837 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.259801 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2"] Jul 09 09:55:16.295216 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.295157 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l"] Jul 09 09:55:16.297198 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:55:16.297161 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9cbbd8d_4aad_4f42_9a60_31c56912a936.slice/crio-7580c54175541250ecbc163732c10986fdaeac3e0bc90a66b0861f6a8defcd84 WatchSource:0}: Error finding container 7580c54175541250ecbc163732c10986fdaeac3e0bc90a66b0861f6a8defcd84: Status 404 returned error can't find the container with id 7580c54175541250ecbc163732c10986fdaeac3e0bc90a66b0861f6a8defcd84 Jul 09 09:55:16.309262 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.309207 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k"] Jul 09 09:55:16.311889 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:55:16.311862 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode49bf34e_61ba_4df7_9daf_4ec806b9b455.slice/crio-516364ea0b8c276455be29803e4f91ea03d9a221ba2a69f5280245ad6b0caa03 WatchSource:0}: Error finding container 516364ea0b8c276455be29803e4f91ea03d9a221ba2a69f5280245ad6b0caa03: Status 404 returned error can't find the container with id 516364ea0b8c276455be29803e4f91ea03d9a221ba2a69f5280245ad6b0caa03 Jul 09 09:55:16.369930 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.369891 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74df7fc877-768cf"] Jul 09 09:55:16.374310 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:55:16.374270 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38e5e107_6837_4790_a6ba_1f9d2c44c725.slice/crio-7df73ee8d15d4137503c337058d4512b9798329383928121d9584fd55fd437b1 WatchSource:0}: Error finding container 7df73ee8d15d4137503c337058d4512b9798329383928121d9584fd55fd437b1: Status 404 returned error can't find the container with id 7df73ee8d15d4137503c337058d4512b9798329383928121d9584fd55fd437b1 Jul 09 09:55:16.425965 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.425932 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2" event={"ID":"58cfca84-a233-4ff9-b44f-1142e072a68b","Type":"ContainerStarted","Data":"bed5e3b6b31cd12d94817290d61b7b9a274fdcaa9ee1a2735d0c0de1f496a804"} Jul 09 09:55:16.427105 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.427076 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l" event={"ID":"b9cbbd8d-4aad-4f42-9a60-31c56912a936","Type":"ContainerStarted","Data":"7580c54175541250ecbc163732c10986fdaeac3e0bc90a66b0861f6a8defcd84"} Jul 09 09:55:16.428672 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.428646 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74df7fc877-768cf" event={"ID":"38e5e107-6837-4790-a6ba-1f9d2c44c725","Type":"ContainerStarted","Data":"7df73ee8d15d4137503c337058d4512b9798329383928121d9584fd55fd437b1"} Jul 09 09:55:16.429763 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:16.429743 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" event={"ID":"e49bf34e-61ba-4df7-9daf-4ec806b9b455","Type":"ContainerStarted","Data":"516364ea0b8c276455be29803e4f91ea03d9a221ba2a69f5280245ad6b0caa03"} Jul 09 09:55:17.437837 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:17.436960 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74df7fc877-768cf" event={"ID":"38e5e107-6837-4790-a6ba-1f9d2c44c725","Type":"ContainerStarted","Data":"8eae6d03b793e6cae4136445c468e1c9e46230487be3d99e4a6d69058638c945"} Jul 09 09:55:17.437837 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:17.437789 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:17.477139 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:17.476460 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74df7fc877-768cf" podStartSLOduration=2.476438027 podStartE2EDuration="2.476438027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:17.475345329 +0000 UTC m=+64.734295193" watchObservedRunningTime="2026-07-09 09:55:17.476438027 +0000 UTC m=+64.735387869" Jul 09 09:55:18.407800 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.406875 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6znj8" Jul 09 09:55:18.448424 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.448382 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2" event={"ID":"58cfca84-a233-4ff9-b44f-1142e072a68b","Type":"ContainerStarted","Data":"1b6411a20fd490f171d096c7bdf48bf9bc3b8daaaa071943daebd4a4d9eceab4"} Jul 09 09:55:18.466921 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.466784 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-6ndl2" podStartSLOduration=2.104332854 podStartE2EDuration="3.466760307s" totalImagesPullingTime="1.362427453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:18.465463372 +0000 UTC m=+65.724413232" watchObservedRunningTime="2026-07-09 09:55:18.466760307 +0000 UTC m=+65.725710168" Jul 09 09:55:18.672553 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.672404 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:55:18.675414 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.675378 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 09:55:18.686157 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.686133 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3cadc2-2f43-4d65-95a2-d83bae26ec1e-metrics-certs\") pod \"network-metrics-daemon-7fx9w\" (UID: \"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e\") " pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:55:18.762795 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.762547 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ffmqr\"" Jul 09 09:55:18.769859 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.769824 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7fx9w" Jul 09 09:55:18.773916 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.773534 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:55:18.777784 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.777576 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 09:55:18.786791 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.786768 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e0bd29c0-1d95-4097-9628-94da4639bcd5-original-pull-secret\") pod \"global-pull-secret-syncer-f987x\" (UID: \"e0bd29c0-1d95-4097-9628-94da4639bcd5\") " pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:55:18.874415 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.874379 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57g88\" (UniqueName: \"kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88\") pod \"network-check-target-fg9gs\" (UID: \"42a23145-ce54-4f13-856c-b112538017d0\") " pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:55:18.877706 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.877677 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 09:55:18.888655 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.888630 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 09:55:18.899300 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:18.899274 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57g88\" (UniqueName: \"kubernetes.io/projected/42a23145-ce54-4f13-856c-b112538017d0-kube-api-access-57g88\") pod \"network-check-target-fg9gs\" (UID: \"42a23145-ce54-4f13-856c-b112538017d0\") " pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:55:19.048013 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:19.047979 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f987x" Jul 09 09:55:19.056942 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:19.056914 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4794j\"" Jul 09 09:55:19.064241 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:19.064218 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:55:20.588695 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:20.588629 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:55:20.591136 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:20.591066 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8b9d4efd-4145-49fd-a355-042b72301086-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ngzg4\" (UID: \"8b9d4efd-4145-49fd-a355-042b72301086\") " pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:55:20.599310 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:20.599223 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fg9gs"] Jul 09 09:55:20.603166 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:55:20.603136 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a23145_ce54_4f13_856c_b112538017d0.slice/crio-76e6c1d6ba9c305cb1b65196f007aa70e10f894ebc822f788f77f7e5a5cd0fb7 WatchSource:0}: Error finding container 76e6c1d6ba9c305cb1b65196f007aa70e10f894ebc822f788f77f7e5a5cd0fb7: Status 404 returned error can't find the container with id 76e6c1d6ba9c305cb1b65196f007aa70e10f894ebc822f788f77f7e5a5cd0fb7 Jul 09 09:55:20.619160 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:20.619131 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7fx9w"] Jul 09 09:55:20.621826 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:55:20.621800 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab3cadc2_2f43_4d65_95a2_d83bae26ec1e.slice/crio-c009eaedd2e5e0c02a9b68f0627bce00b46e7b238815a1ca98398dc88210e1d9 WatchSource:0}: Error finding container c009eaedd2e5e0c02a9b68f0627bce00b46e7b238815a1ca98398dc88210e1d9: Status 404 returned error can't find the container with id c009eaedd2e5e0c02a9b68f0627bce00b46e7b238815a1ca98398dc88210e1d9 Jul 09 09:55:20.779046 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:20.779010 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zkpxd\"" Jul 09 09:55:20.787140 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:20.787117 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ngzg4" Jul 09 09:55:20.821811 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:20.821781 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f987x"] Jul 09 09:55:20.824493 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:55:20.824462 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0bd29c0_1d95_4097_9628_94da4639bcd5.slice/crio-03d6ccc6b23963cc64685bdecd3286f8c8799cfe9dd9aa75338c73de8e9672c3 WatchSource:0}: Error finding container 03d6ccc6b23963cc64685bdecd3286f8c8799cfe9dd9aa75338c73de8e9672c3: Status 404 returned error can't find the container with id 03d6ccc6b23963cc64685bdecd3286f8c8799cfe9dd9aa75338c73de8e9672c3 Jul 09 09:55:20.930831 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:20.930810 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ngzg4"] Jul 09 09:55:20.933160 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:55:20.933131 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b9d4efd_4145_49fd_a355_042b72301086.slice/crio-77a4b9ce322f2ce9ec80894d0598701296176a4c7de53634a6fa9981b6f0ead7 WatchSource:0}: Error finding container 77a4b9ce322f2ce9ec80894d0598701296176a4c7de53634a6fa9981b6f0ead7: Status 404 returned error can't find the container with id 77a4b9ce322f2ce9ec80894d0598701296176a4c7de53634a6fa9981b6f0ead7 Jul 09 09:55:21.466257 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:21.466218 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ngzg4" event={"ID":"8b9d4efd-4145-49fd-a355-042b72301086","Type":"ContainerStarted","Data":"77a4b9ce322f2ce9ec80894d0598701296176a4c7de53634a6fa9981b6f0ead7"} Jul 09 09:55:21.467741 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:21.467709 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7fx9w" event={"ID":"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e","Type":"ContainerStarted","Data":"c009eaedd2e5e0c02a9b68f0627bce00b46e7b238815a1ca98398dc88210e1d9"} Jul 09 09:55:21.469083 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:21.469057 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f987x" event={"ID":"e0bd29c0-1d95-4097-9628-94da4639bcd5","Type":"ContainerStarted","Data":"03d6ccc6b23963cc64685bdecd3286f8c8799cfe9dd9aa75338c73de8e9672c3"} Jul 09 09:55:21.471851 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:21.471823 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" event={"ID":"e49bf34e-61ba-4df7-9daf-4ec806b9b455","Type":"ContainerStarted","Data":"a1fa367dab44af84e112fdbad01625a224f722f2c3743df3f8163b7aaeb66160"} Jul 09 09:55:21.471941 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:21.471863 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" event={"ID":"e49bf34e-61ba-4df7-9daf-4ec806b9b455","Type":"ContainerStarted","Data":"b84a11e85a345ac113a760900572a0988f301f895b98dbce538491a21c36ea4e"} Jul 09 09:55:21.471941 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:21.471884 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" event={"ID":"e49bf34e-61ba-4df7-9daf-4ec806b9b455","Type":"ContainerStarted","Data":"5450c52303f2168a19db2d370295883f4f841873723cc6da21d2e2c42290ff2b"} Jul 09 09:55:21.474243 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:21.474214 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l" event={"ID":"b9cbbd8d-4aad-4f42-9a60-31c56912a936","Type":"ContainerStarted","Data":"3e74de6673424798f98595c9e98f761eaf1dc0da5996a7b5d978396f6e97416b"} Jul 09 09:55:21.475878 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:21.475845 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fg9gs" event={"ID":"42a23145-ce54-4f13-856c-b112538017d0","Type":"ContainerStarted","Data":"76e6c1d6ba9c305cb1b65196f007aa70e10f894ebc822f788f77f7e5a5cd0fb7"} Jul 09 09:55:21.494894 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:21.494831 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7b86d8b489-l7m6k" podStartSLOduration=2.347988133 podStartE2EDuration="6.494810207s" totalImagesPullingTime="4.146822074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:21.493840181 +0000 UTC m=+68.752790041" watchObservedRunningTime="2026-07-09 09:55:21.494810207 +0000 UTC m=+68.753760047" Jul 09 09:55:23.487035 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:23.486995 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7fx9w" event={"ID":"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e","Type":"ContainerStarted","Data":"1b1281a9cfcb9861308f428fc4ec882dfd60e59e6d09fa0555516466768c0e49"} Jul 09 09:55:25.873499 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:25.873469 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:55:25.902017 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:25.901967 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74669d5c94-ltv2l" podStartSLOduration=6.762167527 podStartE2EDuration="10.901947124s" totalImagesPullingTime="4.139779597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:55:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:21.51327744 +0000 UTC m=+68.772227300" watchObservedRunningTime="2026-07-09 09:55:25.901947124 +0000 UTC m=+73.160896982" Jul 09 09:55:28.508036 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:28.507993 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fg9gs" event={"ID":"42a23145-ce54-4f13-856c-b112538017d0","Type":"ContainerStarted","Data":"ef9192e6548edcee25e9a399f96dc449e92637b5134dfa23ee6a5d32af1c40da"} Jul 09 09:55:28.508494 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:28.508097 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:55:28.509537 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:28.509512 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ngzg4" event={"ID":"8b9d4efd-4145-49fd-a355-042b72301086","Type":"ContainerStarted","Data":"bdfd27516adeaf7dae7b9497ae4bfb896e8fdea0c895be23a84d06ce45392fc4"} Jul 09 09:55:28.511242 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:28.511214 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7fx9w" event={"ID":"ab3cadc2-2f43-4d65-95a2-d83bae26ec1e","Type":"ContainerStarted","Data":"ed1ab5166efcb21a54917537a6fa90fbbf1be64f5d7f37c25e8607744172b6c5"} Jul 09 09:55:28.512749 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:28.512724 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f987x" event={"ID":"e0bd29c0-1d95-4097-9628-94da4639bcd5","Type":"ContainerStarted","Data":"f3665401ca50f22450d5f830c7ef234618fbaa38981044d6da59b5ddf1eab9c9"} Jul 09 09:55:28.526650 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:28.526575 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fg9gs" podStartSLOduration=67.127669338 podStartE2EDuration="1m14.526558759s" totalImagesPullingTime="7.398889421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:28.525976711 +0000 UTC m=+75.784926568" watchObservedRunningTime="2026-07-09 09:55:28.526558759 +0000 UTC m=+75.785508621" Jul 09 09:55:28.542967 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:28.542914 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f987x" podStartSLOduration=67.363834373 podStartE2EDuration="1m14.542901017s" totalImagesPullingTime="7.179066644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:28.542692442 +0000 UTC m=+75.801642301" watchObservedRunningTime="2026-07-09 09:55:28.542901017 +0000 UTC m=+75.801850877" Jul 09 09:55:28.563531 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:28.563483 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7fx9w" podStartSLOduration=72.014542019 podStartE2EDuration="1m14.563468942s" totalImagesPullingTime="2.548926923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:54:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:28.56286663 +0000 UTC m=+75.821816489" watchObservedRunningTime="2026-07-09 09:55:28.563468942 +0000 UTC m=+75.822418808" Jul 09 09:55:29.518337 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:29.518293 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ngzg4" event={"ID":"8b9d4efd-4145-49fd-a355-042b72301086","Type":"ContainerStarted","Data":"dd617dd4f859a39bbfb05509a6a0acebcfd07a5db14e7b5d7527f48454166c18"} Jul 09 09:55:31.527701 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:31.527665 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ngzg4" event={"ID":"8b9d4efd-4145-49fd-a355-042b72301086","Type":"ContainerStarted","Data":"0cbe311684a7b394b5d8daefcef48ab4dafde7e37a30c1cc8058eece7071ecd6"} Jul 09 09:55:31.550284 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:31.550231 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ngzg4" podStartSLOduration=33.264863884 podStartE2EDuration="43.550212536s" totalImagesPullingTime="10.285348652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 09:54:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:31.549376749 +0000 UTC m=+78.808326631" watchObservedRunningTime="2026-07-09 09:55:31.550212536 +0000 UTC m=+78.809162396" Jul 09 09:55:32.075815 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.075784 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m64qn"] Jul 09 09:55:32.079317 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.079294 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.082194 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.082169 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 09:55:32.082414 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.082382 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 09:55:32.082544 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.082272 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 09:55:32.082544 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.082289 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 09:55:32.082544 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.082329 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ffn49\"" Jul 09 09:55:32.082729 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.082350 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 09:55:32.082729 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.082194 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 09:55:32.184931 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.184895 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-sys\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.185133 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.184941 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-root\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.185133 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.185014 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-metrics-client-ca\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.185133 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.185093 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-tls\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.185288 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.185135 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.185288 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.185165 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xln6b\" (UniqueName: \"kubernetes.io/projected/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-kube-api-access-xln6b\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.185288 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.185196 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-wtmp\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.185288 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.185275 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-textfile\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.185484 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.185314 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-accelerators-collector-config\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.286034 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.286000 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-tls\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.286271 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.286251 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.286380 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.286367 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xln6b\" (UniqueName: \"kubernetes.io/projected/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-kube-api-access-xln6b\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.286479 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.286467 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-wtmp\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.286575 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.286564 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-textfile\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.286697 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.286681 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-accelerators-collector-config\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.286807 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.286796 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-sys\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.286895 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.286884 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-root\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.286995 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.286984 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-metrics-client-ca\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.287681 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.287652 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-textfile\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.287790 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:55:32.286309 2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 09 09:55:32.287790 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:55:32.287753 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-tls podName:b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d nodeName:}" failed. No retries permitted until 2026-07-09 09:55:32.787732031 +0000 UTC m=+80.046681874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-tls") pod "node-exporter-m64qn" (UID: "b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d") : secret "node-exporter-tls" not found Jul 09 09:55:32.287967 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.287950 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-metrics-client-ca\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.288307 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.288284 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-wtmp\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.288376 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.288350 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-sys\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.288438 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.288390 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-root\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.288699 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.288681 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-accelerators-collector-config\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.297240 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.296988 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.305561 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.305505 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xln6b\" (UniqueName: \"kubernetes.io/projected/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-kube-api-access-xln6b\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.791287 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.791254 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-tls\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.793522 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.793503 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d-node-exporter-tls\") pod \"node-exporter-m64qn\" (UID: \"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d\") " pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:32.989105 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:32.989075 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m64qn" Jul 09 09:55:33.007880 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:55:33.007838 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb73ee8ba_d3aa_4669_a3bf_c5b1261cfe0d.slice/crio-57606f03ec887a1d832f2a0a8d5fd6929da9e7691755d370e047600ed7a16d92 WatchSource:0}: Error finding container 57606f03ec887a1d832f2a0a8d5fd6929da9e7691755d370e047600ed7a16d92: Status 404 returned error can't find the container with id 57606f03ec887a1d832f2a0a8d5fd6929da9e7691755d370e047600ed7a16d92 Jul 09 09:55:33.535738 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:33.535697 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m64qn" event={"ID":"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d","Type":"ContainerStarted","Data":"57606f03ec887a1d832f2a0a8d5fd6929da9e7691755d370e047600ed7a16d92"} Jul 09 09:55:34.540491 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:34.540454 2582 generic.go:356] "Generic (PLEG): container finished" podID="b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d" containerID="e4d04f12303b435d2bf8a4deaac622aeac437ba03ae18d4552dd876c5d1e7a87" exitCode=0 Jul 09 09:55:34.540491 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:34.540495 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m64qn" event={"ID":"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d","Type":"ContainerDied","Data":"e4d04f12303b435d2bf8a4deaac622aeac437ba03ae18d4552dd876c5d1e7a87"} Jul 09 09:55:35.546668 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:35.546630 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m64qn" event={"ID":"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d","Type":"ContainerStarted","Data":"7182b6d78f6032d0d26a7c09f71771d32c55161136eedbd2ed5331785dcd1006"} Jul 09 09:55:35.547053 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:35.546710 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m64qn" event={"ID":"b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d","Type":"ContainerStarted","Data":"7fc09ae5592f331a59caf4476b9f6983d8f61d0520037c964eaec803a13402d0"} Jul 09 09:55:35.569716 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:35.569669 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m64qn" podStartSLOduration=2.5037838470000002 podStartE2EDuration="3.569654721s" totalImagesPullingTime="1.002921141s" totalInitContainerRuntime="62.949733ms" isStatefulPod=true podCreationTimestamp="2026-07-09 09:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:55:35.568891895 +0000 UTC m=+82.827841753" watchObservedRunningTime="2026-07-09 09:55:35.569654721 +0000 UTC m=+82.828604579" Jul 09 09:55:39.458498 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:39.458470 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74df7fc877-768cf" Jul 09 09:55:40.887432 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:40.887371 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" podUID="752bfa27-9e1f-438c-a490-70458fd60715" containerName="registry" containerID="cri-o://dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881" gracePeriod=30 Jul 09 09:55:41.125437 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.125409 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:55:41.256962 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.256926 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/752bfa27-9e1f-438c-a490-70458fd60715-registry-certificates\") pod \"752bfa27-9e1f-438c-a490-70458fd60715\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " Jul 09 09:55:41.256962 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.256979 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/752bfa27-9e1f-438c-a490-70458fd60715-ca-trust-extracted\") pod \"752bfa27-9e1f-438c-a490-70458fd60715\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " Jul 09 09:55:41.257186 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.257002 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls\") pod \"752bfa27-9e1f-438c-a490-70458fd60715\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " Jul 09 09:55:41.257186 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.257036 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-bound-sa-token\") pod \"752bfa27-9e1f-438c-a490-70458fd60715\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " Jul 09 09:55:41.257186 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.257056 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzcvr\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-kube-api-access-jzcvr\") pod \"752bfa27-9e1f-438c-a490-70458fd60715\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " Jul 09 09:55:41.257186 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.257077 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/752bfa27-9e1f-438c-a490-70458fd60715-installation-pull-secrets\") pod \"752bfa27-9e1f-438c-a490-70458fd60715\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " Jul 09 09:55:41.257352 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.257227 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/752bfa27-9e1f-438c-a490-70458fd60715-image-registry-private-configuration\") pod \"752bfa27-9e1f-438c-a490-70458fd60715\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " Jul 09 09:55:41.257352 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.257274 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/752bfa27-9e1f-438c-a490-70458fd60715-trusted-ca\") pod \"752bfa27-9e1f-438c-a490-70458fd60715\" (UID: \"752bfa27-9e1f-438c-a490-70458fd60715\") " Jul 09 09:55:41.257458 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.257395 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/752bfa27-9e1f-438c-a490-70458fd60715-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "752bfa27-9e1f-438c-a490-70458fd60715" (UID: "752bfa27-9e1f-438c-a490-70458fd60715"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 09:55:41.257537 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.257517 2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/752bfa27-9e1f-438c-a490-70458fd60715-registry-certificates\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 09:55:41.257846 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.257800 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/752bfa27-9e1f-438c-a490-70458fd60715-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "752bfa27-9e1f-438c-a490-70458fd60715" (UID: "752bfa27-9e1f-438c-a490-70458fd60715"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 09:55:41.259936 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.259904 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/752bfa27-9e1f-438c-a490-70458fd60715-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "752bfa27-9e1f-438c-a490-70458fd60715" (UID: "752bfa27-9e1f-438c-a490-70458fd60715"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:55:41.260078 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.259950 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "752bfa27-9e1f-438c-a490-70458fd60715" (UID: "752bfa27-9e1f-438c-a490-70458fd60715"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 09:55:41.260078 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.260006 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "752bfa27-9e1f-438c-a490-70458fd60715" (UID: "752bfa27-9e1f-438c-a490-70458fd60715"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 09:55:41.260195 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.260095 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-kube-api-access-jzcvr" (OuterVolumeSpecName: "kube-api-access-jzcvr") pod "752bfa27-9e1f-438c-a490-70458fd60715" (UID: "752bfa27-9e1f-438c-a490-70458fd60715"). InnerVolumeSpecName "kube-api-access-jzcvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 09:55:41.260366 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.260337 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/752bfa27-9e1f-438c-a490-70458fd60715-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "752bfa27-9e1f-438c-a490-70458fd60715" (UID: "752bfa27-9e1f-438c-a490-70458fd60715"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 09:55:41.266115 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.266090 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/752bfa27-9e1f-438c-a490-70458fd60715-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "752bfa27-9e1f-438c-a490-70458fd60715" (UID: "752bfa27-9e1f-438c-a490-70458fd60715"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 09:55:41.358634 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.358584 2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/752bfa27-9e1f-438c-a490-70458fd60715-trusted-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 09:55:41.358634 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.358637 2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/752bfa27-9e1f-438c-a490-70458fd60715-ca-trust-extracted\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 09:55:41.358862 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.358652 2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-registry-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 09:55:41.358862 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.358664 2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-bound-sa-token\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 09:55:41.358862 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.358672 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzcvr\" (UniqueName: \"kubernetes.io/projected/752bfa27-9e1f-438c-a490-70458fd60715-kube-api-access-jzcvr\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 09:55:41.358862 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.358681 2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/752bfa27-9e1f-438c-a490-70458fd60715-installation-pull-secrets\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 09:55:41.358862 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.358691 2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/752bfa27-9e1f-438c-a490-70458fd60715-image-registry-private-configuration\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 09:55:41.568526 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.568447 2582 generic.go:356] "Generic (PLEG): container finished" podID="752bfa27-9e1f-438c-a490-70458fd60715" containerID="dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881" exitCode=0 Jul 09 09:55:41.568526 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.568504 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" Jul 09 09:55:41.568526 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.568516 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" event={"ID":"752bfa27-9e1f-438c-a490-70458fd60715","Type":"ContainerDied","Data":"dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881"} Jul 09 09:55:41.568756 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.568556 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-99b7b67cd-m9x6s" event={"ID":"752bfa27-9e1f-438c-a490-70458fd60715","Type":"ContainerDied","Data":"2d56a5b9b38cdb18f859fc6397f806c2dbb8deb36e44a13fd76c02c12b0f9ae9"} Jul 09 09:55:41.568756 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.568587 2582 scope.go:117] "RemoveContainer" containerID="dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881" Jul 09 09:55:41.576605 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.576587 2582 scope.go:117] "RemoveContainer" containerID="dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881" Jul 09 09:55:41.576919 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:55:41.576901 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881\": container with ID starting with dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881 not found: ID does not exist" containerID="dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881" Jul 09 09:55:41.576978 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.576927 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881"} err="failed to get container status \"dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881\": rpc error: code = NotFound desc = could not find container \"dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881\": container with ID starting with dad1b13d33d87adbd81900e35d0042cdb5e618e95b9a66cc1800a755e3995881 not found: ID does not exist" Jul 09 09:55:41.591313 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.591291 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-99b7b67cd-m9x6s"] Jul 09 09:55:41.596130 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:41.596110 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-99b7b67cd-m9x6s"] Jul 09 09:55:42.039239 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:42.039197 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="752bfa27-9e1f-438c-a490-70458fd60715" path="/var/lib/kubelet/pods/752bfa27-9e1f-438c-a490-70458fd60715/volumes" Jul 09 09:55:59.521313 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:55:59.521278 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fg9gs" Jul 09 09:58:47.420936 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.420857 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9"] Jul 09 09:58:47.421391 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.421113 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="752bfa27-9e1f-438c-a490-70458fd60715" containerName="registry" Jul 09 09:58:47.421391 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.421125 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="752bfa27-9e1f-438c-a490-70458fd60715" containerName="registry" Jul 09 09:58:47.421391 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.421167 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="752bfa27-9e1f-438c-a490-70458fd60715" containerName="registry" Jul 09 09:58:47.423970 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.423951 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.433604 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:58:47.433571 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" type="*v1.Secret" Jul 09 09:58:47.433604 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:58:47.433578 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-dockercfg-zsbc7\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zsbc7\"" type="*v1.Secret" Jul 09 09:58:47.434773 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:58:47.434753 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 09 09:58:47.435166 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:58:47.435146 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" podUID="ed024081-9c20-476f-9e32-dd81e3cb50ea" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.436719 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:58:47.436698 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 09 09:58:47.442066 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.442052 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 09:58:47.455414 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.455387 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed024081-9c20-476f-9e32-dd81e3cb50ea-tmp\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.455521 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.455456 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ed024081-9c20-476f-9e32-dd81e3cb50ea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.455521 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.455474 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed024081-9c20-476f-9e32-dd81e3cb50ea-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.455521 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.455490 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ed024081-9c20-476f-9e32-dd81e3cb50ea-manifests\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.455521 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.455506 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4msqj\" (UniqueName: \"kubernetes.io/projected/ed024081-9c20-476f-9e32-dd81e3cb50ea-kube-api-access-4msqj\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.484140 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.484106 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9"] Jul 09 09:58:47.556398 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.556358 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed024081-9c20-476f-9e32-dd81e3cb50ea-tmp\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.556644 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.556424 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ed024081-9c20-476f-9e32-dd81e3cb50ea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.556644 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.556445 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed024081-9c20-476f-9e32-dd81e3cb50ea-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.556644 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.556463 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ed024081-9c20-476f-9e32-dd81e3cb50ea-manifests\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.556644 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.556479 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4msqj\" (UniqueName: \"kubernetes.io/projected/ed024081-9c20-476f-9e32-dd81e3cb50ea-kube-api-access-4msqj\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.557031 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.556999 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ed024081-9c20-476f-9e32-dd81e3cb50ea-manifests\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.558890 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.558872 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ed024081-9c20-476f-9e32-dd81e3cb50ea-tmp\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.559220 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.559197 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ed024081-9c20-476f-9e32-dd81e3cb50ea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:47.559260 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:47.559208 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed024081-9c20-476f-9e32-dd81e3cb50ea-webhook-cert\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:48.329514 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:48.329480 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 09:58:48.568318 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:58:48.568276 2582 projected.go:291] Couldn't get configMap opendatahub/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 09 09:58:48.568318 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:58:48.568325 2582 projected.go:196] Error preparing data for projected volume kube-api-access-4msqj for pod opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9: failed to sync configmap cache: timed out waiting for the condition Jul 09 09:58:48.568858 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:58:48.568437 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ed024081-9c20-476f-9e32-dd81e3cb50ea-kube-api-access-4msqj podName:ed024081-9c20-476f-9e32-dd81e3cb50ea nodeName:}" failed. No retries permitted until 2026-07-09 09:58:49.068413278 +0000 UTC m=+276.327363119 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4msqj" (UniqueName: "kubernetes.io/projected/ed024081-9c20-476f-9e32-dd81e3cb50ea-kube-api-access-4msqj") pod "opendatahub-operator-controller-manager-5cb598dddb-g6hv9" (UID: "ed024081-9c20-476f-9e32-dd81e3cb50ea") : failed to sync configmap cache: timed out waiting for the condition Jul 09 09:58:48.679566 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:48.679477 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 09:58:48.944848 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:48.944758 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 09:58:48.991606 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:48.991580 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zsbc7\"" Jul 09 09:58:49.069009 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:49.068969 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4msqj\" (UniqueName: \"kubernetes.io/projected/ed024081-9c20-476f-9e32-dd81e3cb50ea-kube-api-access-4msqj\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:49.071433 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:49.071403 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4msqj\" (UniqueName: \"kubernetes.io/projected/ed024081-9c20-476f-9e32-dd81e3cb50ea-kube-api-access-4msqj\") pod \"opendatahub-operator-controller-manager-5cb598dddb-g6hv9\" (UID: \"ed024081-9c20-476f-9e32-dd81e3cb50ea\") " pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:49.233704 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:49.233670 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:49.367561 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:49.367527 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9"] Jul 09 09:58:49.370948 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:58:49.370921 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded024081_9c20_476f_9e32_dd81e3cb50ea.slice/crio-081099a2027e7a5e6441fd8228cbfa184c6d70cfca92c9475ba226ec59733406 WatchSource:0}: Error finding container 081099a2027e7a5e6441fd8228cbfa184c6d70cfca92c9475ba226ec59733406: Status 404 returned error can't find the container with id 081099a2027e7a5e6441fd8228cbfa184c6d70cfca92c9475ba226ec59733406 Jul 09 09:58:50.176374 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:50.176333 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" event={"ID":"ed024081-9c20-476f-9e32-dd81e3cb50ea","Type":"ContainerStarted","Data":"081099a2027e7a5e6441fd8228cbfa184c6d70cfca92c9475ba226ec59733406"} Jul 09 09:58:53.188529 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:53.188488 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" event={"ID":"ed024081-9c20-476f-9e32-dd81e3cb50ea","Type":"ContainerStarted","Data":"8e4b462051e471be2fbd78184e311cbc5af0570c858f3c9b29ac79c2ded0bedf"} Jul 09 09:58:54.193008 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:54.192921 2582 generic.go:356] "Generic (PLEG): container finished" podID="ed024081-9c20-476f-9e32-dd81e3cb50ea" containerID="8e4b462051e471be2fbd78184e311cbc5af0570c858f3c9b29ac79c2ded0bedf" exitCode=0 Jul 09 09:58:54.193343 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:54.193006 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" event={"ID":"ed024081-9c20-476f-9e32-dd81e3cb50ea","Type":"ContainerDied","Data":"8e4b462051e471be2fbd78184e311cbc5af0570c858f3c9b29ac79c2ded0bedf"} Jul 09 09:58:55.197806 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:55.197773 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" event={"ID":"ed024081-9c20-476f-9e32-dd81e3cb50ea","Type":"ContainerStarted","Data":"cb64da04d0c85d543f89052ee4a53a11a7996b8365a3a92b1aed4614ebd2dcbf"} Jul 09 09:58:55.198184 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:55.197920 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:58:55.225861 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:58:55.225807 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" podStartSLOduration=3.622684745 podStartE2EDuration="8.225790519s" totalImagesPullingTime="3.570658s" totalInitContainerRuntime="1.032447774s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:58:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:58:55.22381249 +0000 UTC m=+282.482762346" watchObservedRunningTime="2026-07-09 09:58:55.225790519 +0000 UTC m=+282.484740377" Jul 09 09:59:06.203632 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:06.203590 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5cb598dddb-g6hv9" Jul 09 09:59:09.386370 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.386334 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-589b555f76-xkg8s"] Jul 09 09:59:09.388908 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.388888 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:09.396462 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:59:09.396435 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-creds\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" type="*v1.Secret" Jul 09 09:59:09.400583 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:59:09.400557 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kube-auth-proxy-dockercfg-8rgdb\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8rgdb\"" type="*v1.Secret" Jul 09 09:59:09.400947 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.400926 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 09:59:09.400947 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:59:09.400929 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kube-auth-proxy-589b555f76-xkg8s\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" podUID="c62438ba-e90e-4dc9-93f9-69115eab1e43" pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:09.401077 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:59:09.400922 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-134-220.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-134-220.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 09 09:59:09.404187 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.404170 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 09:59:09.428350 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.428315 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-589b555f76-xkg8s"] Jul 09 09:59:09.528354 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.528323 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c62438ba-e90e-4dc9-93f9-69115eab1e43-tls-certs\") pod \"kube-auth-proxy-589b555f76-xkg8s\" (UID: \"c62438ba-e90e-4dc9-93f9-69115eab1e43\") " pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:09.528498 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.528377 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c62438ba-e90e-4dc9-93f9-69115eab1e43-tmp\") pod \"kube-auth-proxy-589b555f76-xkg8s\" (UID: \"c62438ba-e90e-4dc9-93f9-69115eab1e43\") " pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:09.528498 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.528441 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mpwh\" (UniqueName: \"kubernetes.io/projected/c62438ba-e90e-4dc9-93f9-69115eab1e43-kube-api-access-2mpwh\") pod \"kube-auth-proxy-589b555f76-xkg8s\" (UID: \"c62438ba-e90e-4dc9-93f9-69115eab1e43\") " pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:09.629148 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.629107 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c62438ba-e90e-4dc9-93f9-69115eab1e43-tmp\") pod \"kube-auth-proxy-589b555f76-xkg8s\" (UID: \"c62438ba-e90e-4dc9-93f9-69115eab1e43\") " pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:09.629345 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.629162 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mpwh\" (UniqueName: \"kubernetes.io/projected/c62438ba-e90e-4dc9-93f9-69115eab1e43-kube-api-access-2mpwh\") pod \"kube-auth-proxy-589b555f76-xkg8s\" (UID: \"c62438ba-e90e-4dc9-93f9-69115eab1e43\") " pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:09.629345 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.629193 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c62438ba-e90e-4dc9-93f9-69115eab1e43-tls-certs\") pod \"kube-auth-proxy-589b555f76-xkg8s\" (UID: \"c62438ba-e90e-4dc9-93f9-69115eab1e43\") " pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:09.631490 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.631463 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c62438ba-e90e-4dc9-93f9-69115eab1e43-tmp\") pod \"kube-auth-proxy-589b555f76-xkg8s\" (UID: \"c62438ba-e90e-4dc9-93f9-69115eab1e43\") " pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:09.631647 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:09.631606 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c62438ba-e90e-4dc9-93f9-69115eab1e43-tls-certs\") pod \"kube-auth-proxy-589b555f76-xkg8s\" (UID: \"c62438ba-e90e-4dc9-93f9-69115eab1e43\") " pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:10.349972 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:10.349937 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 09:59:10.524888 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:10.524852 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-8rgdb\"" Jul 09 09:59:10.644900 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:59:10.644809 2582 projected.go:291] Couldn't get configMap openshift-ingress/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 09 09:59:10.644900 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:59:10.644850 2582 projected.go:196] Error preparing data for projected volume kube-api-access-2mpwh for pod openshift-ingress/kube-auth-proxy-589b555f76-xkg8s: failed to sync configmap cache: timed out waiting for the condition Jul 09 09:59:10.645088 ip-10-0-134-220 kubenswrapper[2582]: E0709 09:59:10.644921 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c62438ba-e90e-4dc9-93f9-69115eab1e43-kube-api-access-2mpwh podName:c62438ba-e90e-4dc9-93f9-69115eab1e43 nodeName:}" failed. No retries permitted until 2026-07-09 09:59:11.144903773 +0000 UTC m=+298.403853622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2mpwh" (UniqueName: "kubernetes.io/projected/c62438ba-e90e-4dc9-93f9-69115eab1e43-kube-api-access-2mpwh") pod "kube-auth-proxy-589b555f76-xkg8s" (UID: "c62438ba-e90e-4dc9-93f9-69115eab1e43") : failed to sync configmap cache: timed out waiting for the condition Jul 09 09:59:10.763967 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:10.763933 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 09:59:10.995809 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:10.995764 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5pk4g"] Jul 09 09:59:10.998107 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:10.998083 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" Jul 09 09:59:11.001755 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.001734 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 09 09:59:11.033104 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.033081 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5pk4g"] Jul 09 09:59:11.142217 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.142180 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5lvq\" (UniqueName: \"kubernetes.io/projected/04beab64-df60-4159-bfb8-07abe4f1c3b4-kube-api-access-n5lvq\") pod \"model-serving-api-54d89bd5fc-5pk4g\" (UID: \"04beab64-df60-4159-bfb8-07abe4f1c3b4\") " pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" Jul 09 09:59:11.142347 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.142240 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04beab64-df60-4159-bfb8-07abe4f1c3b4-tls-certs\") pod \"model-serving-api-54d89bd5fc-5pk4g\" (UID: \"04beab64-df60-4159-bfb8-07abe4f1c3b4\") " pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" Jul 09 09:59:11.243311 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.243275 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5lvq\" (UniqueName: \"kubernetes.io/projected/04beab64-df60-4159-bfb8-07abe4f1c3b4-kube-api-access-n5lvq\") pod \"model-serving-api-54d89bd5fc-5pk4g\" (UID: \"04beab64-df60-4159-bfb8-07abe4f1c3b4\") " pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" Jul 09 09:59:11.243455 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.243328 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mpwh\" (UniqueName: \"kubernetes.io/projected/c62438ba-e90e-4dc9-93f9-69115eab1e43-kube-api-access-2mpwh\") pod \"kube-auth-proxy-589b555f76-xkg8s\" (UID: \"c62438ba-e90e-4dc9-93f9-69115eab1e43\") " pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:11.243455 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.243376 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04beab64-df60-4159-bfb8-07abe4f1c3b4-tls-certs\") pod \"model-serving-api-54d89bd5fc-5pk4g\" (UID: \"04beab64-df60-4159-bfb8-07abe4f1c3b4\") " pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" Jul 09 09:59:11.245785 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.245763 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04beab64-df60-4159-bfb8-07abe4f1c3b4-tls-certs\") pod \"model-serving-api-54d89bd5fc-5pk4g\" (UID: \"04beab64-df60-4159-bfb8-07abe4f1c3b4\") " pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" Jul 09 09:59:11.245920 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.245855 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mpwh\" (UniqueName: \"kubernetes.io/projected/c62438ba-e90e-4dc9-93f9-69115eab1e43-kube-api-access-2mpwh\") pod \"kube-auth-proxy-589b555f76-xkg8s\" (UID: \"c62438ba-e90e-4dc9-93f9-69115eab1e43\") " pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:11.266328 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.266299 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5lvq\" (UniqueName: \"kubernetes.io/projected/04beab64-df60-4159-bfb8-07abe4f1c3b4-kube-api-access-n5lvq\") pod \"model-serving-api-54d89bd5fc-5pk4g\" (UID: \"04beab64-df60-4159-bfb8-07abe4f1c3b4\") " pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" Jul 09 09:59:11.308443 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.308414 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" Jul 09 09:59:11.455332 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.455257 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5pk4g"] Jul 09 09:59:11.457891 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:59:11.457866 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04beab64_df60_4159_bfb8_07abe4f1c3b4.slice/crio-86d5555316a098453680b9bebbf599a2a882ea943f15b845fa43c1b54714f63e WatchSource:0}: Error finding container 86d5555316a098453680b9bebbf599a2a882ea943f15b845fa43c1b54714f63e: Status 404 returned error can't find the container with id 86d5555316a098453680b9bebbf599a2a882ea943f15b845fa43c1b54714f63e Jul 09 09:59:11.497587 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.497518 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" Jul 09 09:59:11.640963 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:11.640939 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-589b555f76-xkg8s"] Jul 09 09:59:11.642845 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:59:11.642817 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc62438ba_e90e_4dc9_93f9_69115eab1e43.slice/crio-098e827c013503b3d87eac16d9d436619774ec47e4ba2395f661dbe0eccf8fef WatchSource:0}: Error finding container 098e827c013503b3d87eac16d9d436619774ec47e4ba2395f661dbe0eccf8fef: Status 404 returned error can't find the container with id 098e827c013503b3d87eac16d9d436619774ec47e4ba2395f661dbe0eccf8fef Jul 09 09:59:12.256814 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:12.256740 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" event={"ID":"c62438ba-e90e-4dc9-93f9-69115eab1e43","Type":"ContainerStarted","Data":"098e827c013503b3d87eac16d9d436619774ec47e4ba2395f661dbe0eccf8fef"} Jul 09 09:59:12.258235 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:12.258210 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" event={"ID":"04beab64-df60-4159-bfb8-07abe4f1c3b4","Type":"ContainerStarted","Data":"86d5555316a098453680b9bebbf599a2a882ea943f15b845fa43c1b54714f63e"} Jul 09 09:59:13.997371 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:13.997347 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 09:59:14.001478 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:14.001458 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 09:59:14.268315 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:14.268212 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" event={"ID":"c62438ba-e90e-4dc9-93f9-69115eab1e43","Type":"ContainerStarted","Data":"8cbc5fe87e2e398b5dbd6d790651f48cd1888bd5d1de13decc77bd96634cc403"} Jul 09 09:59:14.270512 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:14.269637 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" event={"ID":"04beab64-df60-4159-bfb8-07abe4f1c3b4","Type":"ContainerStarted","Data":"a7d2d8287ccdbbf231e0047ae567c9e5bb06f172fe2114c0f3983fd2c4128a1b"} Jul 09 09:59:14.270512 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:14.269780 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" Jul 09 09:59:14.370899 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:14.370845 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" podStartSLOduration=1.8482691039999999 podStartE2EDuration="4.370828626s" totalImagesPullingTime="2.522559522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:59:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:59:14.368014594 +0000 UTC m=+301.626964453" watchObservedRunningTime="2026-07-09 09:59:14.370828626 +0000 UTC m=+301.629778485" Jul 09 09:59:14.371293 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:14.371259 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-589b555f76-xkg8s" podStartSLOduration=3.022752813 podStartE2EDuration="5.371248118s" totalImagesPullingTime="2.348495305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:59:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:59:14.315467231 +0000 UTC m=+301.574417090" watchObservedRunningTime="2026-07-09 09:59:14.371248118 +0000 UTC m=+301.630198000" Jul 09 09:59:15.797924 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.797891 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk"] Jul 09 09:59:15.800948 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.800931 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:15.812692 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.812669 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 09:59:15.812794 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.812706 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 09:59:15.812794 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.812755 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 09:59:15.813011 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.812995 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 09:59:15.813526 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.813513 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lkvqp\"" Jul 09 09:59:15.814735 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.814721 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 09:59:15.840253 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.840231 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk"] Jul 09 09:59:15.879308 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.879280 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd5b19a7-4cff-43b4-bb5a-34c526b62cb1-cert\") pod \"lws-controller-manager-5c844974c9-l8kvk\" (UID: \"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:15.879444 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.879322 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cd5b19a7-4cff-43b4-bb5a-34c526b62cb1-manager-config\") pod \"lws-controller-manager-5c844974c9-l8kvk\" (UID: \"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:15.879444 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.879355 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrswt\" (UniqueName: \"kubernetes.io/projected/cd5b19a7-4cff-43b4-bb5a-34c526b62cb1-kube-api-access-mrswt\") pod \"lws-controller-manager-5c844974c9-l8kvk\" (UID: \"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:15.879444 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.879412 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cd5b19a7-4cff-43b4-bb5a-34c526b62cb1-metrics-cert\") pod \"lws-controller-manager-5c844974c9-l8kvk\" (UID: \"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:15.980480 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.980445 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cd5b19a7-4cff-43b4-bb5a-34c526b62cb1-metrics-cert\") pod \"lws-controller-manager-5c844974c9-l8kvk\" (UID: \"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:15.980705 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.980507 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd5b19a7-4cff-43b4-bb5a-34c526b62cb1-cert\") pod \"lws-controller-manager-5c844974c9-l8kvk\" (UID: \"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:15.980705 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.980548 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cd5b19a7-4cff-43b4-bb5a-34c526b62cb1-manager-config\") pod \"lws-controller-manager-5c844974c9-l8kvk\" (UID: \"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:15.980705 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.980576 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrswt\" (UniqueName: \"kubernetes.io/projected/cd5b19a7-4cff-43b4-bb5a-34c526b62cb1-kube-api-access-mrswt\") pod \"lws-controller-manager-5c844974c9-l8kvk\" (UID: \"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:15.981222 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.981202 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/cd5b19a7-4cff-43b4-bb5a-34c526b62cb1-manager-config\") pod \"lws-controller-manager-5c844974c9-l8kvk\" (UID: \"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:15.982908 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.982889 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/cd5b19a7-4cff-43b4-bb5a-34c526b62cb1-metrics-cert\") pod \"lws-controller-manager-5c844974c9-l8kvk\" (UID: \"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:15.982968 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:15.982930 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd5b19a7-4cff-43b4-bb5a-34c526b62cb1-cert\") pod \"lws-controller-manager-5c844974c9-l8kvk\" (UID: \"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:16.008093 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:16.008057 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrswt\" (UniqueName: \"kubernetes.io/projected/cd5b19a7-4cff-43b4-bb5a-34c526b62cb1-kube-api-access-mrswt\") pod \"lws-controller-manager-5c844974c9-l8kvk\" (UID: \"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1\") " pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:16.109846 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:16.109749 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:16.489155 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:16.489115 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk"] Jul 09 09:59:16.494789 ip-10-0-134-220 kubenswrapper[2582]: W0709 09:59:16.494762 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd5b19a7_4cff_43b4_bb5a_34c526b62cb1.slice/crio-02e5ba74b64c8d6d83c055e70f66ecbedd3404f6fb95edaad14f5553039baba9 WatchSource:0}: Error finding container 02e5ba74b64c8d6d83c055e70f66ecbedd3404f6fb95edaad14f5553039baba9: Status 404 returned error can't find the container with id 02e5ba74b64c8d6d83c055e70f66ecbedd3404f6fb95edaad14f5553039baba9 Jul 09 09:59:16.496656 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:16.496640 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 09:59:17.283448 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:17.283372 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" event={"ID":"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1","Type":"ContainerStarted","Data":"02e5ba74b64c8d6d83c055e70f66ecbedd3404f6fb95edaad14f5553039baba9"} Jul 09 09:59:19.292237 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:19.292201 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" event={"ID":"cd5b19a7-4cff-43b4-bb5a-34c526b62cb1","Type":"ContainerStarted","Data":"9bb20175f735a200085f3e43d8b6a21bf869afd353ab02eeb39aa690d5e39cff"} Jul 09 09:59:19.292713 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:19.292346 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 09:59:25.278042 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:25.278013 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-5pk4g" Jul 09 09:59:25.368004 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:25.367947 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" podStartSLOduration=8.041980574 podStartE2EDuration="10.367932422s" totalImagesPullingTime="2.325951848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 09:59:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 09:59:19.35906292 +0000 UTC m=+306.618012778" watchObservedRunningTime="2026-07-09 09:59:25.367932422 +0000 UTC m=+312.626882282" Jul 09 09:59:30.298634 ip-10-0-134-220 kubenswrapper[2582]: I0709 09:59:30.298584 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c844974c9-l8kvk" Jul 09 10:00:54.437062 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.437025 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5"] Jul 09 10:00:54.440097 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.440078 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" Jul 09 10:00:54.445228 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.445205 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 09 10:00:54.445337 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.445246 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 10:00:54.445400 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.445375 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zvqhc\"" Jul 09 10:00:54.457162 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.457134 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 10:00:54.457162 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.457134 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 09 10:00:54.508848 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.508816 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5"] Jul 09 10:00:54.544564 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.544534 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/78daf914-2660-4943-8122-d501ee70fc07-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-c7sc5\" (UID: \"78daf914-2660-4943-8122-d501ee70fc07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" Jul 09 10:00:54.544754 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.544570 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/78daf914-2660-4943-8122-d501ee70fc07-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-c7sc5\" (UID: \"78daf914-2660-4943-8122-d501ee70fc07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" Jul 09 10:00:54.544754 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.544663 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t28ff\" (UniqueName: \"kubernetes.io/projected/78daf914-2660-4943-8122-d501ee70fc07-kube-api-access-t28ff\") pod \"kuadrant-console-plugin-66b5774985-c7sc5\" (UID: \"78daf914-2660-4943-8122-d501ee70fc07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" Jul 09 10:00:54.645470 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.645430 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t28ff\" (UniqueName: \"kubernetes.io/projected/78daf914-2660-4943-8122-d501ee70fc07-kube-api-access-t28ff\") pod \"kuadrant-console-plugin-66b5774985-c7sc5\" (UID: \"78daf914-2660-4943-8122-d501ee70fc07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" Jul 09 10:00:54.645722 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.645486 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/78daf914-2660-4943-8122-d501ee70fc07-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-c7sc5\" (UID: \"78daf914-2660-4943-8122-d501ee70fc07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" Jul 09 10:00:54.645722 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.645509 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/78daf914-2660-4943-8122-d501ee70fc07-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-c7sc5\" (UID: \"78daf914-2660-4943-8122-d501ee70fc07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" Jul 09 10:00:54.646232 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.646206 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/78daf914-2660-4943-8122-d501ee70fc07-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-c7sc5\" (UID: \"78daf914-2660-4943-8122-d501ee70fc07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" Jul 09 10:00:54.647966 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.647935 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/78daf914-2660-4943-8122-d501ee70fc07-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-c7sc5\" (UID: \"78daf914-2660-4943-8122-d501ee70fc07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" Jul 09 10:00:54.664995 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.664959 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t28ff\" (UniqueName: \"kubernetes.io/projected/78daf914-2660-4943-8122-d501ee70fc07-kube-api-access-t28ff\") pod \"kuadrant-console-plugin-66b5774985-c7sc5\" (UID: \"78daf914-2660-4943-8122-d501ee70fc07\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" Jul 09 10:00:54.748591 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.748558 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" Jul 09 10:00:54.896348 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:54.896323 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5"] Jul 09 10:00:54.898931 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:00:54.898906 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78daf914_2660_4943_8122_d501ee70fc07.slice/crio-5ffcf1a7a96ce98786f0cf537e6161c4c5d726fcb410f6d9e67552c2a8e862c0 WatchSource:0}: Error finding container 5ffcf1a7a96ce98786f0cf537e6161c4c5d726fcb410f6d9e67552c2a8e862c0: Status 404 returned error can't find the container with id 5ffcf1a7a96ce98786f0cf537e6161c4c5d726fcb410f6d9e67552c2a8e862c0 Jul 09 10:00:55.640506 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:00:55.640462 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" event={"ID":"78daf914-2660-4943-8122-d501ee70fc07","Type":"ContainerStarted","Data":"5ffcf1a7a96ce98786f0cf537e6161c4c5d726fcb410f6d9e67552c2a8e862c0"} Jul 09 10:01:17.742415 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:17.742373 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" event={"ID":"78daf914-2660-4943-8122-d501ee70fc07","Type":"ContainerStarted","Data":"4e714eee188aba52d912204287c2fabc21c087d878b7f4c7465a35761305a8d8"} Jul 09 10:01:17.760117 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:17.760067 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-c7sc5" podStartSLOduration=1.8433181159999998 podStartE2EDuration="23.760051299s" totalImagesPullingTime="21.916733183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:00:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:01:17.758957339 +0000 UTC m=+425.017907197" watchObservedRunningTime="2026-07-09 10:01:17.760051299 +0000 UTC m=+425.019001157" Jul 09 10:01:37.423476 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:37.423381 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-fvfcn"] Jul 09 10:01:37.438567 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:37.438539 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fvfcn"] Jul 09 10:01:37.438737 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:37.438650 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fvfcn" Jul 09 10:01:37.441406 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:37.441376 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rvd7h\"" Jul 09 10:01:37.606063 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:37.606020 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr5j4\" (UniqueName: \"kubernetes.io/projected/f373a7b3-bcff-4136-bc19-573560d7a976-kube-api-access-sr5j4\") pod \"authorino-75d556bd77-fvfcn\" (UID: \"f373a7b3-bcff-4136-bc19-573560d7a976\") " pod="kuadrant-system/authorino-75d556bd77-fvfcn" Jul 09 10:01:37.707069 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:37.706990 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr5j4\" (UniqueName: \"kubernetes.io/projected/f373a7b3-bcff-4136-bc19-573560d7a976-kube-api-access-sr5j4\") pod \"authorino-75d556bd77-fvfcn\" (UID: \"f373a7b3-bcff-4136-bc19-573560d7a976\") " pod="kuadrant-system/authorino-75d556bd77-fvfcn" Jul 09 10:01:37.717063 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:37.717033 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr5j4\" (UniqueName: \"kubernetes.io/projected/f373a7b3-bcff-4136-bc19-573560d7a976-kube-api-access-sr5j4\") pod \"authorino-75d556bd77-fvfcn\" (UID: \"f373a7b3-bcff-4136-bc19-573560d7a976\") " pod="kuadrant-system/authorino-75d556bd77-fvfcn" Jul 09 10:01:37.747965 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:37.747917 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fvfcn" Jul 09 10:01:37.890293 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:37.890270 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fvfcn"] Jul 09 10:01:37.892522 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:01:37.892494 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf373a7b3_bcff_4136_bc19_573560d7a976.slice/crio-e9cbda6f395099bb512c7ffe0dcd1a7e40ce623eb8a24737a8f276ab21cd4984 WatchSource:0}: Error finding container e9cbda6f395099bb512c7ffe0dcd1a7e40ce623eb8a24737a8f276ab21cd4984: Status 404 returned error can't find the container with id e9cbda6f395099bb512c7ffe0dcd1a7e40ce623eb8a24737a8f276ab21cd4984 Jul 09 10:01:38.832852 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:38.832806 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fvfcn" event={"ID":"f373a7b3-bcff-4136-bc19-573560d7a976","Type":"ContainerStarted","Data":"e9cbda6f395099bb512c7ffe0dcd1a7e40ce623eb8a24737a8f276ab21cd4984"} Jul 09 10:01:42.851979 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:42.851939 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fvfcn" event={"ID":"f373a7b3-bcff-4136-bc19-573560d7a976","Type":"ContainerStarted","Data":"48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6"} Jul 09 10:01:42.869233 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:01:42.869181 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-fvfcn" podStartSLOduration=1.676629683 podStartE2EDuration="5.869165665s" totalImagesPullingTime="4.192535982s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:01:42.867965373 +0000 UTC m=+450.126915232" watchObservedRunningTime="2026-07-09 10:01:42.869165665 +0000 UTC m=+450.128115524" Jul 09 10:02:13.616389 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:13.616353 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fvfcn"] Jul 09 10:02:13.616942 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:13.616524 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-fvfcn" podUID="f373a7b3-bcff-4136-bc19-573560d7a976" containerName="authorino" containerID="cri-o://48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6" gracePeriod=30 Jul 09 10:02:13.889815 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:13.889745 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fvfcn" Jul 09 10:02:13.973315 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:13.973276 2582 generic.go:356] "Generic (PLEG): container finished" podID="f373a7b3-bcff-4136-bc19-573560d7a976" containerID="48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6" exitCode=0 Jul 09 10:02:13.973478 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:13.973320 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-fvfcn" Jul 09 10:02:13.973478 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:13.973361 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fvfcn" event={"ID":"f373a7b3-bcff-4136-bc19-573560d7a976","Type":"ContainerDied","Data":"48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6"} Jul 09 10:02:13.973478 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:13.973402 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-fvfcn" event={"ID":"f373a7b3-bcff-4136-bc19-573560d7a976","Type":"ContainerDied","Data":"e9cbda6f395099bb512c7ffe0dcd1a7e40ce623eb8a24737a8f276ab21cd4984"} Jul 09 10:02:13.973478 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:13.973421 2582 scope.go:117] "RemoveContainer" containerID="48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6" Jul 09 10:02:13.981025 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:13.981009 2582 scope.go:117] "RemoveContainer" containerID="48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6" Jul 09 10:02:13.981310 ip-10-0-134-220 kubenswrapper[2582]: E0709 10:02:13.981292 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6\": container with ID starting with 48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6 not found: ID does not exist" containerID="48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6" Jul 09 10:02:13.981365 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:13.981322 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6"} err="failed to get container status \"48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6\": rpc error: code = NotFound desc = could not find container \"48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6\": container with ID starting with 48290026962a646671b16af015b4c675898015a000b15fffad9d6cf1df86d0e6 not found: ID does not exist" Jul 09 10:02:13.981365 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:13.981329 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr5j4\" (UniqueName: \"kubernetes.io/projected/f373a7b3-bcff-4136-bc19-573560d7a976-kube-api-access-sr5j4\") pod \"f373a7b3-bcff-4136-bc19-573560d7a976\" (UID: \"f373a7b3-bcff-4136-bc19-573560d7a976\") " Jul 09 10:02:13.983431 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:13.983409 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f373a7b3-bcff-4136-bc19-573560d7a976-kube-api-access-sr5j4" (OuterVolumeSpecName: "kube-api-access-sr5j4") pod "f373a7b3-bcff-4136-bc19-573560d7a976" (UID: "f373a7b3-bcff-4136-bc19-573560d7a976"). InnerVolumeSpecName "kube-api-access-sr5j4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:02:14.082340 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:14.082305 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr5j4\" (UniqueName: \"kubernetes.io/projected/f373a7b3-bcff-4136-bc19-573560d7a976-kube-api-access-sr5j4\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:02:14.297326 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:14.297291 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fvfcn"] Jul 09 10:02:14.314276 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:14.314245 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-fvfcn"] Jul 09 10:02:16.040262 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:02:16.040220 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f373a7b3-bcff-4136-bc19-573560d7a976" path="/var/lib/kubelet/pods/f373a7b3-bcff-4136-bc19-573560d7a976/volumes" Jul 09 10:03:21.491528 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.491497 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm"] Jul 09 10:03:21.491939 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.491782 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f373a7b3-bcff-4136-bc19-573560d7a976" containerName="authorino" Jul 09 10:03:21.491939 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.491794 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f373a7b3-bcff-4136-bc19-573560d7a976" containerName="authorino" Jul 09 10:03:21.491939 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.491852 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f373a7b3-bcff-4136-bc19-573560d7a976" containerName="authorino" Jul 09 10:03:21.495745 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.495728 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.499558 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.499535 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 10:03:21.499711 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.499535 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xzxkm\"" Jul 09 10:03:21.499711 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.499538 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 09 10:03:21.499711 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.499584 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 10:03:21.509581 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.509556 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm"] Jul 09 10:03:21.584309 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.584278 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2307e6f2-49fc-4055-a874-100bb8757ee9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.584483 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.584333 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2307e6f2-49fc-4055-a874-100bb8757ee9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.584483 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.584357 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2307e6f2-49fc-4055-a874-100bb8757ee9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.584483 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.584382 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2307e6f2-49fc-4055-a874-100bb8757ee9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.584483 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.584398 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l4sp\" (UniqueName: \"kubernetes.io/projected/2307e6f2-49fc-4055-a874-100bb8757ee9-kube-api-access-9l4sp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.584483 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.584413 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2307e6f2-49fc-4055-a874-100bb8757ee9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.684776 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.684739 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2307e6f2-49fc-4055-a874-100bb8757ee9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.684776 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.684782 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2307e6f2-49fc-4055-a874-100bb8757ee9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.684977 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.684799 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2307e6f2-49fc-4055-a874-100bb8757ee9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.684977 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.684820 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l4sp\" (UniqueName: \"kubernetes.io/projected/2307e6f2-49fc-4055-a874-100bb8757ee9-kube-api-access-9l4sp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.684977 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.684852 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2307e6f2-49fc-4055-a874-100bb8757ee9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.684977 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.684895 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2307e6f2-49fc-4055-a874-100bb8757ee9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.685198 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.685173 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2307e6f2-49fc-4055-a874-100bb8757ee9-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.685240 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.685212 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2307e6f2-49fc-4055-a874-100bb8757ee9-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.685280 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.685247 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2307e6f2-49fc-4055-a874-100bb8757ee9-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.687149 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.687127 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2307e6f2-49fc-4055-a874-100bb8757ee9-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.687386 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.687369 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2307e6f2-49fc-4055-a874-100bb8757ee9-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.695515 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.695487 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l4sp\" (UniqueName: \"kubernetes.io/projected/2307e6f2-49fc-4055-a874-100bb8757ee9-kube-api-access-9l4sp\") pod \"e2e-distinct-simulated-kserve-59975d69f9-4njhm\" (UID: \"2307e6f2-49fc-4055-a874-100bb8757ee9\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.806647 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.806508 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:21.936266 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:21.936243 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm"] Jul 09 10:03:21.938760 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:03:21.938732 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2307e6f2_49fc_4055_a874_100bb8757ee9.slice/crio-19721befcca19cadec5c8e2f4fe4eb3906d83130e29045a55a42098d3be1af53 WatchSource:0}: Error finding container 19721befcca19cadec5c8e2f4fe4eb3906d83130e29045a55a42098d3be1af53: Status 404 returned error can't find the container with id 19721befcca19cadec5c8e2f4fe4eb3906d83130e29045a55a42098d3be1af53 Jul 09 10:03:22.235086 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:22.235046 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" event={"ID":"2307e6f2-49fc-4055-a874-100bb8757ee9","Type":"ContainerStarted","Data":"19721befcca19cadec5c8e2f4fe4eb3906d83130e29045a55a42098d3be1af53"} Jul 09 10:03:27.255769 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:27.255733 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" event={"ID":"2307e6f2-49fc-4055-a874-100bb8757ee9","Type":"ContainerStarted","Data":"d9cebb6318711b3c37f2656be2594d024a76229ebec6a9195dbc007497c8b5de"} Jul 09 10:03:27.279559 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:27.279512 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" podStartSLOduration=1.437335566 podStartE2EDuration="6.279497306s" totalImagesPullingTime="4.84216174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:03:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:03:27.278921526 +0000 UTC m=+554.537871384" watchObservedRunningTime="2026-07-09 10:03:27.279497306 +0000 UTC m=+554.538447164" Jul 09 10:03:31.807301 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:31.807261 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:31.807803 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:31.807324 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:31.820070 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:31.820050 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:32.285390 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:32.285360 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-4njhm" Jul 09 10:03:35.705608 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.705573 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs"] Jul 09 10:03:35.708877 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.708861 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.712236 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.712213 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 09 10:03:35.721634 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.721596 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs"] Jul 09 10:03:35.779815 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.779775 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87d9d933-22f4-4c09-95bd-cb4908653c49-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.779968 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.779830 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87d9d933-22f4-4c09-95bd-cb4908653c49-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.779968 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.779852 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87d9d933-22f4-4c09-95bd-cb4908653c49-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.779968 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.779892 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87d9d933-22f4-4c09-95bd-cb4908653c49-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.779968 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.779910 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcgc5\" (UniqueName: \"kubernetes.io/projected/87d9d933-22f4-4c09-95bd-cb4908653c49-kube-api-access-zcgc5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.780098 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.779962 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87d9d933-22f4-4c09-95bd-cb4908653c49-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.880407 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.880365 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87d9d933-22f4-4c09-95bd-cb4908653c49-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.880407 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.880408 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcgc5\" (UniqueName: \"kubernetes.io/projected/87d9d933-22f4-4c09-95bd-cb4908653c49-kube-api-access-zcgc5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.880677 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.880440 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87d9d933-22f4-4c09-95bd-cb4908653c49-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.880677 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.880485 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87d9d933-22f4-4c09-95bd-cb4908653c49-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.880677 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.880546 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87d9d933-22f4-4c09-95bd-cb4908653c49-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.880677 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.880576 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87d9d933-22f4-4c09-95bd-cb4908653c49-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.880842 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.880820 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87d9d933-22f4-4c09-95bd-cb4908653c49-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.880945 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.880923 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87d9d933-22f4-4c09-95bd-cb4908653c49-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.880980 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.880931 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87d9d933-22f4-4c09-95bd-cb4908653c49-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.882881 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.882853 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87d9d933-22f4-4c09-95bd-cb4908653c49-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.883135 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.883115 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87d9d933-22f4-4c09-95bd-cb4908653c49-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:35.891279 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:35.891252 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcgc5\" (UniqueName: \"kubernetes.io/projected/87d9d933-22f4-4c09-95bd-cb4908653c49-kube-api-access-zcgc5\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs\" (UID: \"87d9d933-22f4-4c09-95bd-cb4908653c49\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:36.019363 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:36.019318 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:36.148789 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:36.148694 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs"] Jul 09 10:03:36.150941 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:03:36.150908 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87d9d933_22f4_4c09_95bd_cb4908653c49.slice/crio-d25d0520e2667bc68ca9b6e87f5212ea33dd8aad0101dfb5973296863164bb64 WatchSource:0}: Error finding container d25d0520e2667bc68ca9b6e87f5212ea33dd8aad0101dfb5973296863164bb64: Status 404 returned error can't find the container with id d25d0520e2667bc68ca9b6e87f5212ea33dd8aad0101dfb5973296863164bb64 Jul 09 10:03:36.290391 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:36.290309 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" event={"ID":"87d9d933-22f4-4c09-95bd-cb4908653c49","Type":"ContainerStarted","Data":"d25d0520e2667bc68ca9b6e87f5212ea33dd8aad0101dfb5973296863164bb64"} Jul 09 10:03:37.296989 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:37.296954 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" event={"ID":"87d9d933-22f4-4c09-95bd-cb4908653c49","Type":"ContainerStarted","Data":"d27b311a5e9d03d20353d2f66ce348dfe8ba3d9de725373130fce9e3333ec9b8"} Jul 09 10:03:37.322488 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:37.322442 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" podStartSLOduration=2.018061506 podStartE2EDuration="2.322415964s" totalImagesPullingTime="304.354458ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:03:37.321517412 +0000 UTC m=+564.580467270" watchObservedRunningTime="2026-07-09 10:03:37.322415964 +0000 UTC m=+564.581365823" Jul 09 10:03:46.020190 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:46.020152 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:46.020586 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:46.020285 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:46.032455 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:46.032427 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:46.341641 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:46.341543 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs" Jul 09 10:03:51.392771 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.392722 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm"] Jul 09 10:03:51.396058 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.396035 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.399074 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.399052 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 09 10:03:51.411319 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.411297 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm"] Jul 09 10:03:51.509936 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.509898 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.509936 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.509942 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.510157 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.509965 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.510157 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.510027 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.510157 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.510078 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.510157 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.510100 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfckd\" (UniqueName: \"kubernetes.io/projected/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-kube-api-access-rfckd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.610752 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.610706 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.610935 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.610762 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfckd\" (UniqueName: \"kubernetes.io/projected/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-kube-api-access-rfckd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.610935 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.610818 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.610935 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.610852 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.610935 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.610882 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.610935 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.610918 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.611235 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.611207 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.611297 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.611213 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.611297 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.611261 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.612854 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.612832 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.613375 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.613354 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.621555 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.621534 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfckd\" (UniqueName: \"kubernetes.io/projected/21771a3d-e377-42dd-91d3-5a99b6bc2c6a-kube-api-access-rfckd\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm\" (UID: \"21771a3d-e377-42dd-91d3-5a99b6bc2c6a\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.704833 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.704758 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:03:51.852004 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:51.851963 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm"] Jul 09 10:03:51.854587 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:03:51.854558 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21771a3d_e377_42dd_91d3_5a99b6bc2c6a.slice/crio-9c438af26bceb98ba62eb4ad1575e99b587dfa6e63419cdb95a866ca293c3788 WatchSource:0}: Error finding container 9c438af26bceb98ba62eb4ad1575e99b587dfa6e63419cdb95a866ca293c3788: Status 404 returned error can't find the container with id 9c438af26bceb98ba62eb4ad1575e99b587dfa6e63419cdb95a866ca293c3788 Jul 09 10:03:52.353281 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:52.353248 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" event={"ID":"21771a3d-e377-42dd-91d3-5a99b6bc2c6a","Type":"ContainerStarted","Data":"9c438af26bceb98ba62eb4ad1575e99b587dfa6e63419cdb95a866ca293c3788"} Jul 09 10:03:54.362371 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:54.362334 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" event={"ID":"21771a3d-e377-42dd-91d3-5a99b6bc2c6a","Type":"ContainerStarted","Data":"f8c0cfebd595b6e29c3a2ddad30e14b0c41d2fbb1f043ecf0403d34676ed7772"} Jul 09 10:03:54.387630 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:03:54.387568 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" podStartSLOduration=1.696095189 podStartE2EDuration="3.387551934s" totalImagesPullingTime="1.691456745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:03:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:03:54.385461787 +0000 UTC m=+581.644411645" watchObservedRunningTime="2026-07-09 10:03:54.387551934 +0000 UTC m=+581.646501793" Jul 09 10:04:01.705892 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:01.705843 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:04:01.706375 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:01.706111 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:04:01.718597 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:01.718575 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:04:02.406443 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:02.406413 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm" Jul 09 10:04:38.400744 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.400665 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8588b87745-sjxt9"] Jul 09 10:04:38.404008 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.403991 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8588b87745-sjxt9" Jul 09 10:04:38.407659 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.407638 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rvd7h\"" Jul 09 10:04:38.407781 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.407665 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 09 10:04:38.412016 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.411993 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8588b87745-sjxt9"] Jul 09 10:04:38.489972 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.489936 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtpqp\" (UniqueName: \"kubernetes.io/projected/c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5-kube-api-access-jtpqp\") pod \"authorino-8588b87745-sjxt9\" (UID: \"c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5\") " pod="kuadrant-system/authorino-8588b87745-sjxt9" Jul 09 10:04:38.490132 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.490038 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5-tls-cert\") pod \"authorino-8588b87745-sjxt9\" (UID: \"c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5\") " pod="kuadrant-system/authorino-8588b87745-sjxt9" Jul 09 10:04:38.590607 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.590568 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtpqp\" (UniqueName: \"kubernetes.io/projected/c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5-kube-api-access-jtpqp\") pod \"authorino-8588b87745-sjxt9\" (UID: \"c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5\") " pod="kuadrant-system/authorino-8588b87745-sjxt9" Jul 09 10:04:38.590801 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.590659 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5-tls-cert\") pod \"authorino-8588b87745-sjxt9\" (UID: \"c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5\") " pod="kuadrant-system/authorino-8588b87745-sjxt9" Jul 09 10:04:38.593094 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.593073 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5-tls-cert\") pod \"authorino-8588b87745-sjxt9\" (UID: \"c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5\") " pod="kuadrant-system/authorino-8588b87745-sjxt9" Jul 09 10:04:38.600996 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.600972 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtpqp\" (UniqueName: \"kubernetes.io/projected/c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5-kube-api-access-jtpqp\") pod \"authorino-8588b87745-sjxt9\" (UID: \"c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5\") " pod="kuadrant-system/authorino-8588b87745-sjxt9" Jul 09 10:04:38.714068 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.713995 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8588b87745-sjxt9" Jul 09 10:04:38.842349 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.842326 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8588b87745-sjxt9"] Jul 09 10:04:38.844909 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:04:38.844880 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc94bc8d0_ab3e_4a2c_9a72_c4cde612dfb5.slice/crio-bd92f05a628d56885696120ff23734ed042b3708f0f125be3d60ca1f5c136801 WatchSource:0}: Error finding container bd92f05a628d56885696120ff23734ed042b3708f0f125be3d60ca1f5c136801: Status 404 returned error can't find the container with id bd92f05a628d56885696120ff23734ed042b3708f0f125be3d60ca1f5c136801 Jul 09 10:04:38.846635 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:38.846605 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 10:04:39.552359 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:39.552317 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8588b87745-sjxt9" event={"ID":"c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5","Type":"ContainerStarted","Data":"8ad2f1c3b318972b7c8f4d758da414a05b20bd3691fb60633a7c1562f7f0eba7"} Jul 09 10:04:39.552753 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:39.552371 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8588b87745-sjxt9" event={"ID":"c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5","Type":"ContainerStarted","Data":"bd92f05a628d56885696120ff23734ed042b3708f0f125be3d60ca1f5c136801"} Jul 09 10:04:39.569475 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:04:39.569365 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8588b87745-sjxt9" podStartSLOduration=1.102911254 podStartE2EDuration="1.569349112s" totalImagesPullingTime="466.437858ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:04:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:04:39.567531317 +0000 UTC m=+626.826481177" watchObservedRunningTime="2026-07-09 10:04:39.569349112 +0000 UTC m=+626.828298971" Jul 09 10:21:38.597713 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.597677 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4"] Jul 09 10:21:38.600913 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.600898 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.604650 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.604605 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7dafaaa7\"/\"default-dockercfg-k6jht\"" Jul 09 10:21:38.604780 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.604655 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7dafaaa7\"/\"kube-root-ca.crt\"" Jul 09 10:21:38.604780 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.604756 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7dafaaa7\"/\"openshift-service-ca.crt\"" Jul 09 10:21:38.604898 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.604760 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7dafaaa7\"/\"e2e-lifecycle-model-7dafaaa7-kserve-self-signed-certs\"" Jul 09 10:21:38.611697 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.611677 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4"] Jul 09 10:21:38.671950 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.671918 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x65rz\" (UniqueName: \"kubernetes.io/projected/acdfd32e-8fec-44c6-876d-060d55baf3c8-kube-api-access-x65rz\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.672096 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.671956 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-home\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.672096 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.672036 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-dshm\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.672096 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.672071 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-model-cache\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.672217 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.672111 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-tmp-dir\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.672217 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.672127 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acdfd32e-8fec-44c6-876d-060d55baf3c8-tls-certs\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.772540 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.772489 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-dshm\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.772739 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.772559 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-model-cache\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.772739 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.772587 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-tmp-dir\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.772739 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.772609 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acdfd32e-8fec-44c6-876d-060d55baf3c8-tls-certs\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.772739 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.772691 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x65rz\" (UniqueName: \"kubernetes.io/projected/acdfd32e-8fec-44c6-876d-060d55baf3c8-kube-api-access-x65rz\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.772739 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.772726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-home\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.773024 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.773000 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-model-cache\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.773087 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.773011 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-tmp-dir\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.773134 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.773082 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-home\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.774874 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.774843 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-dshm\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.775043 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.775028 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acdfd32e-8fec-44c6-876d-060d55baf3c8-tls-certs\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.781852 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.781821 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x65rz\" (UniqueName: \"kubernetes.io/projected/acdfd32e-8fec-44c6-876d-060d55baf3c8-kube-api-access-x65rz\") pod \"e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:38.909994 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:38.909912 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:39.036779 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:39.036758 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4"] Jul 09 10:21:39.038676 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:21:39.038648 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacdfd32e_8fec_44c6_876d_060d55baf3c8.slice/crio-fc54a754f0b2d7967f1b645d6cc2eca5cc7bfae807743916cb924f810c519f10 WatchSource:0}: Error finding container fc54a754f0b2d7967f1b645d6cc2eca5cc7bfae807743916cb924f810c519f10: Status 404 returned error can't find the container with id fc54a754f0b2d7967f1b645d6cc2eca5cc7bfae807743916cb924f810c519f10 Jul 09 10:21:39.707138 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:39.707097 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" event={"ID":"acdfd32e-8fec-44c6-876d-060d55baf3c8","Type":"ContainerStarted","Data":"a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427"} Jul 09 10:21:39.707138 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:39.707142 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" event={"ID":"acdfd32e-8fec-44c6-876d-060d55baf3c8","Type":"ContainerStarted","Data":"fc54a754f0b2d7967f1b645d6cc2eca5cc7bfae807743916cb924f810c519f10"} Jul 09 10:21:39.727384 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:39.727331 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" podStartSLOduration=1.727315642 podStartE2EDuration="1.727315642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:21:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:21:39.725264526 +0000 UTC m=+1646.984214408" watchObservedRunningTime="2026-07-09 10:21:39.727315642 +0000 UTC m=+1646.986265506" Jul 09 10:21:48.910246 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:48.910205 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:48.910681 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:48.910263 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:48.922492 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:48.922468 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:21:49.759828 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:21:49.759798 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:22:09.591109 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:09.591078 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4"] Jul 09 10:22:24.610948 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.610877 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" podUID="acdfd32e-8fec-44c6-876d-060d55baf3c8" containerName="main" containerID="cri-o://a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427" gracePeriod=45 Jul 09 10:22:24.849293 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.849269 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:22:24.885591 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.885507 2582 generic.go:356] "Generic (PLEG): container finished" podID="acdfd32e-8fec-44c6-876d-060d55baf3c8" containerID="a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427" exitCode=0 Jul 09 10:22:24.885591 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.885557 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" Jul 09 10:22:24.885757 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.885594 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" event={"ID":"acdfd32e-8fec-44c6-876d-060d55baf3c8","Type":"ContainerDied","Data":"a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427"} Jul 09 10:22:24.885757 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.885653 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4" event={"ID":"acdfd32e-8fec-44c6-876d-060d55baf3c8","Type":"ContainerDied","Data":"fc54a754f0b2d7967f1b645d6cc2eca5cc7bfae807743916cb924f810c519f10"} Jul 09 10:22:24.885757 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.885670 2582 scope.go:117] "RemoveContainer" containerID="a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427" Jul 09 10:22:24.893999 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.893980 2582 scope.go:117] "RemoveContainer" containerID="a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427" Jul 09 10:22:24.894271 ip-10-0-134-220 kubenswrapper[2582]: E0709 10:22:24.894246 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427\": container with ID starting with a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427 not found: ID does not exist" containerID="a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427" Jul 09 10:22:24.894375 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.894276 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427"} err="failed to get container status \"a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427\": rpc error: code = NotFound desc = could not find container \"a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427\": container with ID starting with a42a7b480fb2ee8169d918bb60d7b2e86fec5bb8c5e1b53b56d6f92ce70ea427 not found: ID does not exist" Jul 09 10:22:24.956404 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.956378 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-tmp-dir\") pod \"acdfd32e-8fec-44c6-876d-060d55baf3c8\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " Jul 09 10:22:24.956557 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.956413 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x65rz\" (UniqueName: \"kubernetes.io/projected/acdfd32e-8fec-44c6-876d-060d55baf3c8-kube-api-access-x65rz\") pod \"acdfd32e-8fec-44c6-876d-060d55baf3c8\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " Jul 09 10:22:24.956557 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.956429 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-dshm\") pod \"acdfd32e-8fec-44c6-876d-060d55baf3c8\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " Jul 09 10:22:24.956557 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.956476 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acdfd32e-8fec-44c6-876d-060d55baf3c8-tls-certs\") pod \"acdfd32e-8fec-44c6-876d-060d55baf3c8\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " Jul 09 10:22:24.956557 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.956499 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-home\") pod \"acdfd32e-8fec-44c6-876d-060d55baf3c8\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " Jul 09 10:22:24.956557 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.956529 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-model-cache\") pod \"acdfd32e-8fec-44c6-876d-060d55baf3c8\" (UID: \"acdfd32e-8fec-44c6-876d-060d55baf3c8\") " Jul 09 10:22:24.956838 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.956685 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "acdfd32e-8fec-44c6-876d-060d55baf3c8" (UID: "acdfd32e-8fec-44c6-876d-060d55baf3c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:22:24.956838 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.956796 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-home" (OuterVolumeSpecName: "home") pod "acdfd32e-8fec-44c6-876d-060d55baf3c8" (UID: "acdfd32e-8fec-44c6-876d-060d55baf3c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:22:24.956838 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.956825 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-model-cache" (OuterVolumeSpecName: "model-cache") pod "acdfd32e-8fec-44c6-876d-060d55baf3c8" (UID: "acdfd32e-8fec-44c6-876d-060d55baf3c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:22:24.956947 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.956851 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-tmp-dir\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:22:24.958755 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.958729 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acdfd32e-8fec-44c6-876d-060d55baf3c8-kube-api-access-x65rz" (OuterVolumeSpecName: "kube-api-access-x65rz") pod "acdfd32e-8fec-44c6-876d-060d55baf3c8" (UID: "acdfd32e-8fec-44c6-876d-060d55baf3c8"). InnerVolumeSpecName "kube-api-access-x65rz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:22:24.958832 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.958761 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-dshm" (OuterVolumeSpecName: "dshm") pod "acdfd32e-8fec-44c6-876d-060d55baf3c8" (UID: "acdfd32e-8fec-44c6-876d-060d55baf3c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:22:24.958832 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:24.958769 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acdfd32e-8fec-44c6-876d-060d55baf3c8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "acdfd32e-8fec-44c6-876d-060d55baf3c8" (UID: "acdfd32e-8fec-44c6-876d-060d55baf3c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 10:22:25.057469 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:25.057435 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-dshm\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:22:25.057469 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:25.057463 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acdfd32e-8fec-44c6-876d-060d55baf3c8-tls-certs\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:22:25.057469 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:25.057472 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-home\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:22:25.057713 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:25.057480 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acdfd32e-8fec-44c6-876d-060d55baf3c8-model-cache\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:22:25.057713 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:25.057489 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x65rz\" (UniqueName: \"kubernetes.io/projected/acdfd32e-8fec-44c6-876d-060d55baf3c8-kube-api-access-x65rz\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:22:25.208018 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:25.207987 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4"] Jul 09 10:22:25.212520 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:25.212496 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7dafaaa7/e2e-lifecycle-model-7dafaaa7-kserve-6f578ff4c5-hjqq4"] Jul 09 10:22:26.039773 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:22:26.039741 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acdfd32e-8fec-44c6-876d-060d55baf3c8" path="/var/lib/kubelet/pods/acdfd32e-8fec-44c6-876d-060d55baf3c8/volumes" Jul 09 10:24:08.171015 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.170928 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k"] Jul 09 10:24:08.171533 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.171361 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acdfd32e-8fec-44c6-876d-060d55baf3c8" containerName="main" Jul 09 10:24:08.171533 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.171379 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="acdfd32e-8fec-44c6-876d-060d55baf3c8" containerName="main" Jul 09 10:24:08.171533 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.171460 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="acdfd32e-8fec-44c6-876d-060d55baf3c8" containerName="main" Jul 09 10:24:08.174371 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.174348 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:08.177090 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.177067 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-npsbz\"" Jul 09 10:24:08.177195 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.177097 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 10:24:08.178199 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.178176 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 10:24:08.178322 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.178235 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-14869d-serving-cert\"" Jul 09 10:24:08.186866 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.186847 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k"] Jul 09 10:24:08.281569 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.281525 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wvhq\" (UniqueName: \"kubernetes.io/projected/574fd649-383f-4da9-8372-8cacf1559819-kube-api-access-5wvhq\") pod \"maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k\" (UID: \"574fd649-383f-4da9-8372-8cacf1559819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:08.281757 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.281645 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/574fd649-383f-4da9-8372-8cacf1559819-maas-api-tls\") pod \"maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k\" (UID: \"574fd649-383f-4da9-8372-8cacf1559819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:08.281757 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.281698 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/574fd649-383f-4da9-8372-8cacf1559819-openshift-service-ca\") pod \"maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k\" (UID: \"574fd649-383f-4da9-8372-8cacf1559819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:08.382634 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.382585 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/574fd649-383f-4da9-8372-8cacf1559819-openshift-service-ca\") pod \"maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k\" (UID: \"574fd649-383f-4da9-8372-8cacf1559819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:08.382801 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.382655 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wvhq\" (UniqueName: \"kubernetes.io/projected/574fd649-383f-4da9-8372-8cacf1559819-kube-api-access-5wvhq\") pod \"maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k\" (UID: \"574fd649-383f-4da9-8372-8cacf1559819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:08.382801 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.382705 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/574fd649-383f-4da9-8372-8cacf1559819-maas-api-tls\") pod \"maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k\" (UID: \"574fd649-383f-4da9-8372-8cacf1559819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:08.383234 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.383208 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/574fd649-383f-4da9-8372-8cacf1559819-openshift-service-ca\") pod \"maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k\" (UID: \"574fd649-383f-4da9-8372-8cacf1559819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:08.385075 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.385046 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/574fd649-383f-4da9-8372-8cacf1559819-maas-api-tls\") pod \"maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k\" (UID: \"574fd649-383f-4da9-8372-8cacf1559819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:08.401753 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.401732 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wvhq\" (UniqueName: \"kubernetes.io/projected/574fd649-383f-4da9-8372-8cacf1559819-kube-api-access-5wvhq\") pod \"maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k\" (UID: \"574fd649-383f-4da9-8372-8cacf1559819\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:08.484942 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.484912 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:08.616735 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.616708 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k"] Jul 09 10:24:08.619104 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:24:08.619068 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod574fd649_383f_4da9_8372_8cacf1559819.slice/crio-f628c77931808f07f5fdcff325e13751a44fb54f0348a18fb38e1e74e29cbf07 WatchSource:0}: Error finding container f628c77931808f07f5fdcff325e13751a44fb54f0348a18fb38e1e74e29cbf07: Status 404 returned error can't find the container with id f628c77931808f07f5fdcff325e13751a44fb54f0348a18fb38e1e74e29cbf07 Jul 09 10:24:08.620325 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:08.620307 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 10:24:09.301755 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:09.301718 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" event={"ID":"574fd649-383f-4da9-8372-8cacf1559819","Type":"ContainerStarted","Data":"f628c77931808f07f5fdcff325e13751a44fb54f0348a18fb38e1e74e29cbf07"} Jul 09 10:24:13.326037 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:13.325998 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" event={"ID":"574fd649-383f-4da9-8372-8cacf1559819","Type":"ContainerStarted","Data":"41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a"} Jul 09 10:24:13.326523 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:13.326100 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:13.348068 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:13.348019 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" podStartSLOduration=1.6389395599999999 podStartE2EDuration="5.348004632s" totalImagesPullingTime="3.709065072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:24:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:24:13.346250842 +0000 UTC m=+1800.605200700" watchObservedRunningTime="2026-07-09 10:24:13.348004632 +0000 UTC m=+1800.606954491" Jul 09 10:24:19.335765 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:19.335739 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:24:25.300947 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.300914 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48"] Jul 09 10:24:25.306053 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.306035 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.308981 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.308958 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-31eb62\"/\"openshift-service-ca.crt\"" Jul 09 10:24:25.310327 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.310306 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-31eb62\"/\"test-model-31eb62-kserve-self-signed-certs\"" Jul 09 10:24:25.310493 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.310479 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-31eb62\"/\"default-dockercfg-5qnsr\"" Jul 09 10:24:25.310570 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.310552 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-31eb62\"/\"kube-root-ca.crt\"" Jul 09 10:24:25.317965 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.317943 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48"] Jul 09 10:24:25.428811 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.428774 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-model-cache\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.428811 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.428815 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-dshm\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.429059 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.428836 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-tmp-dir\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.429059 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.428919 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-home\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.429059 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.428955 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4qzz\" (UniqueName: \"kubernetes.io/projected/de29a743-e1d9-4b87-b6ed-1dee9dc64134-kube-api-access-g4qzz\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.429059 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.429045 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de29a743-e1d9-4b87-b6ed-1dee9dc64134-tls-certs\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.530101 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.530064 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-dshm\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.530101 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.530102 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-tmp-dir\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.530365 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.530130 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-home\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.530365 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.530150 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4qzz\" (UniqueName: \"kubernetes.io/projected/de29a743-e1d9-4b87-b6ed-1dee9dc64134-kube-api-access-g4qzz\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.530365 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.530201 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de29a743-e1d9-4b87-b6ed-1dee9dc64134-tls-certs\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.530365 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.530242 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-model-cache\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.530597 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.530571 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-tmp-dir\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.530696 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.530594 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-home\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.530696 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.530679 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-model-cache\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.532506 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.532483 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-dshm\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.532770 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.532751 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de29a743-e1d9-4b87-b6ed-1dee9dc64134-tls-certs\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.543994 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.543974 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4qzz\" (UniqueName: \"kubernetes.io/projected/de29a743-e1d9-4b87-b6ed-1dee9dc64134-kube-api-access-g4qzz\") pod \"test-model-31eb62-kserve-6b94494bf5-j8q48\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.614491 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.614411 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:25.750418 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:25.750392 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48"] Jul 09 10:24:25.752866 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:24:25.752839 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde29a743_e1d9_4b87_b6ed_1dee9dc64134.slice/crio-8756b5ecf4fe978c2811be9250a2bcd0517538976bbd685e426255224a2559c0 WatchSource:0}: Error finding container 8756b5ecf4fe978c2811be9250a2bcd0517538976bbd685e426255224a2559c0: Status 404 returned error can't find the container with id 8756b5ecf4fe978c2811be9250a2bcd0517538976bbd685e426255224a2559c0 Jul 09 10:24:26.384180 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:26.384147 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" event={"ID":"de29a743-e1d9-4b87-b6ed-1dee9dc64134","Type":"ContainerStarted","Data":"c009e24fcf695bb19c5c5f7e0bdd0ce0765ef74b1390d29f55698c7ec4e976a0"} Jul 09 10:24:26.384602 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:26.384198 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" event={"ID":"de29a743-e1d9-4b87-b6ed-1dee9dc64134","Type":"ContainerStarted","Data":"8756b5ecf4fe978c2811be9250a2bcd0517538976bbd685e426255224a2559c0"} Jul 09 10:24:26.403373 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:26.403321 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" podStartSLOduration=1.403306977 podStartE2EDuration="1.403306977s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:24:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:24:26.402301934 +0000 UTC m=+1813.661251798" watchObservedRunningTime="2026-07-09 10:24:26.403306977 +0000 UTC m=+1813.662256836" Jul 09 10:24:35.615552 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:35.615513 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:35.615970 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:35.615569 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:35.628132 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:35.628110 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:36.435579 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:36.435548 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:24:46.589059 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.589027 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn"] Jul 09 10:24:46.591985 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.591968 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.595053 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.595037 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-14869d\"/\"kube-root-ca.crt\"" Jul 09 10:24:46.596180 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.596159 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-14869d\"/\"test-model-14869d-kserve-self-signed-certs\"" Jul 09 10:24:46.596255 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.596168 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-14869d\"/\"default-dockercfg-tn7sq\"" Jul 09 10:24:46.596255 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.596196 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-14869d\"/\"openshift-service-ca.crt\"" Jul 09 10:24:46.603922 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.603897 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn"] Jul 09 10:24:46.716316 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.716276 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxn4h\" (UniqueName: \"kubernetes.io/projected/f7863475-333d-4f6c-a720-aa59b89d99d2-kube-api-access-jxn4h\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.716316 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.716323 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-dshm\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.716569 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.716345 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-tmp-dir\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.716569 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.716441 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-home\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.716569 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.716490 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7863475-333d-4f6c-a720-aa59b89d99d2-tls-certs\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.716710 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.716564 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-model-cache\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.817668 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.817630 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxn4h\" (UniqueName: \"kubernetes.io/projected/f7863475-333d-4f6c-a720-aa59b89d99d2-kube-api-access-jxn4h\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.817837 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.817681 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-dshm\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.817837 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.817793 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-tmp-dir\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.817923 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.817855 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-home\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.817923 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.817887 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7863475-333d-4f6c-a720-aa59b89d99d2-tls-certs\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.817923 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.817912 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-model-cache\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.818274 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.818252 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-home\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.818333 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.818304 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-tmp-dir\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.818367 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.818329 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-model-cache\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.820134 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.820114 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-dshm\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.820396 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.820379 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7863475-333d-4f6c-a720-aa59b89d99d2-tls-certs\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.831343 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.831317 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxn4h\" (UniqueName: \"kubernetes.io/projected/f7863475-333d-4f6c-a720-aa59b89d99d2-kube-api-access-jxn4h\") pod \"test-model-14869d-kserve-d5b7797-89wnn\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:46.901177 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:46.901092 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:47.044365 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:47.044345 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn"] Jul 09 10:24:47.046844 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:24:47.046808 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7863475_333d_4f6c_a720_aa59b89d99d2.slice/crio-bc1aa082204afda2a45bb5dda179925d25a447b788d37ee4564a814fd4e37158 WatchSource:0}: Error finding container bc1aa082204afda2a45bb5dda179925d25a447b788d37ee4564a814fd4e37158: Status 404 returned error can't find the container with id bc1aa082204afda2a45bb5dda179925d25a447b788d37ee4564a814fd4e37158 Jul 09 10:24:47.468582 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:47.468540 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" event={"ID":"f7863475-333d-4f6c-a720-aa59b89d99d2","Type":"ContainerStarted","Data":"fa020320221125d42f13e6e425ab414fa322d56b0b2b44510fe85de9c117207c"} Jul 09 10:24:47.468582 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:47.468586 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" event={"ID":"f7863475-333d-4f6c-a720-aa59b89d99d2","Type":"ContainerStarted","Data":"bc1aa082204afda2a45bb5dda179925d25a447b788d37ee4564a814fd4e37158"} Jul 09 10:24:47.489248 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:47.489199 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" podStartSLOduration=1.489185553 podStartE2EDuration="1.489185553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:24:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:24:47.487419776 +0000 UTC m=+1834.746369676" watchObservedRunningTime="2026-07-09 10:24:47.489185553 +0000 UTC m=+1834.748135453" Jul 09 10:24:56.901812 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:56.901770 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:56.901812 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:56.901821 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:56.914025 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:56.913993 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:24:57.521404 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:24:57.521375 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:25:06.451285 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:06.451254 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48"] Jul 09 10:25:07.469180 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:07.469146 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn"] Jul 09 10:25:21.471229 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.471165 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" podUID="de29a743-e1d9-4b87-b6ed-1dee9dc64134" containerName="main" containerID="cri-o://c009e24fcf695bb19c5c5f7e0bdd0ce0765ef74b1390d29f55698c7ec4e976a0" gracePeriod=45 Jul 09 10:25:21.616553 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.616447 2582 generic.go:356] "Generic (PLEG): container finished" podID="de29a743-e1d9-4b87-b6ed-1dee9dc64134" containerID="c009e24fcf695bb19c5c5f7e0bdd0ce0765ef74b1390d29f55698c7ec4e976a0" exitCode=0 Jul 09 10:25:21.616553 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.616525 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" event={"ID":"de29a743-e1d9-4b87-b6ed-1dee9dc64134","Type":"ContainerDied","Data":"c009e24fcf695bb19c5c5f7e0bdd0ce0765ef74b1390d29f55698c7ec4e976a0"} Jul 09 10:25:21.711196 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.711169 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:25:21.816338 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.816306 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-tmp-dir\") pod \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " Jul 09 10:25:21.816509 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.816368 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-home\") pod \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " Jul 09 10:25:21.816509 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.816399 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-dshm\") pod \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " Jul 09 10:25:21.816509 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.816438 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4qzz\" (UniqueName: \"kubernetes.io/projected/de29a743-e1d9-4b87-b6ed-1dee9dc64134-kube-api-access-g4qzz\") pod \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " Jul 09 10:25:21.816509 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.816494 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de29a743-e1d9-4b87-b6ed-1dee9dc64134-tls-certs\") pod \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " Jul 09 10:25:21.816747 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.816548 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-model-cache\") pod \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\" (UID: \"de29a743-e1d9-4b87-b6ed-1dee9dc64134\") " Jul 09 10:25:21.816747 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.816574 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de29a743-e1d9-4b87-b6ed-1dee9dc64134" (UID: "de29a743-e1d9-4b87-b6ed-1dee9dc64134"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:21.816747 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.816715 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-home" (OuterVolumeSpecName: "home") pod "de29a743-e1d9-4b87-b6ed-1dee9dc64134" (UID: "de29a743-e1d9-4b87-b6ed-1dee9dc64134"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:21.816903 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.816841 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-home\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:21.816903 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.816876 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-model-cache" (OuterVolumeSpecName: "model-cache") pod "de29a743-e1d9-4b87-b6ed-1dee9dc64134" (UID: "de29a743-e1d9-4b87-b6ed-1dee9dc64134"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:21.816903 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.816885 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-tmp-dir\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:21.818689 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.818662 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-dshm" (OuterVolumeSpecName: "dshm") pod "de29a743-e1d9-4b87-b6ed-1dee9dc64134" (UID: "de29a743-e1d9-4b87-b6ed-1dee9dc64134"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:21.818816 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.818754 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de29a743-e1d9-4b87-b6ed-1dee9dc64134-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de29a743-e1d9-4b87-b6ed-1dee9dc64134" (UID: "de29a743-e1d9-4b87-b6ed-1dee9dc64134"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 10:25:21.818816 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.818799 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de29a743-e1d9-4b87-b6ed-1dee9dc64134-kube-api-access-g4qzz" (OuterVolumeSpecName: "kube-api-access-g4qzz") pod "de29a743-e1d9-4b87-b6ed-1dee9dc64134" (UID: "de29a743-e1d9-4b87-b6ed-1dee9dc64134"). InnerVolumeSpecName "kube-api-access-g4qzz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:25:21.917328 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.917290 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-dshm\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:21.917328 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.917324 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4qzz\" (UniqueName: \"kubernetes.io/projected/de29a743-e1d9-4b87-b6ed-1dee9dc64134-kube-api-access-g4qzz\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:21.917328 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.917336 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de29a743-e1d9-4b87-b6ed-1dee9dc64134-tls-certs\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:21.917568 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:21.917345 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de29a743-e1d9-4b87-b6ed-1dee9dc64134-model-cache\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:22.489110 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.489070 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" podUID="f7863475-333d-4f6c-a720-aa59b89d99d2" containerName="main" containerID="cri-o://fa020320221125d42f13e6e425ab414fa322d56b0b2b44510fe85de9c117207c" gracePeriod=45 Jul 09 10:25:22.621602 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.621577 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" Jul 09 10:25:22.621602 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.621592 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48" event={"ID":"de29a743-e1d9-4b87-b6ed-1dee9dc64134","Type":"ContainerDied","Data":"8756b5ecf4fe978c2811be9250a2bcd0517538976bbd685e426255224a2559c0"} Jul 09 10:25:22.621809 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.621665 2582 scope.go:117] "RemoveContainer" containerID="c009e24fcf695bb19c5c5f7e0bdd0ce0765ef74b1390d29f55698c7ec4e976a0" Jul 09 10:25:22.623294 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.623269 2582 generic.go:356] "Generic (PLEG): container finished" podID="f7863475-333d-4f6c-a720-aa59b89d99d2" containerID="fa020320221125d42f13e6e425ab414fa322d56b0b2b44510fe85de9c117207c" exitCode=0 Jul 09 10:25:22.623418 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.623397 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" event={"ID":"f7863475-333d-4f6c-a720-aa59b89d99d2","Type":"ContainerDied","Data":"fa020320221125d42f13e6e425ab414fa322d56b0b2b44510fe85de9c117207c"} Jul 09 10:25:22.640776 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.640746 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48"] Jul 09 10:25:22.646455 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.646430 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-31eb62/test-model-31eb62-kserve-6b94494bf5-j8q48"] Jul 09 10:25:22.733902 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.733871 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:25:22.825260 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.825167 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-model-cache\") pod \"f7863475-333d-4f6c-a720-aa59b89d99d2\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " Jul 09 10:25:22.825260 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.825215 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxn4h\" (UniqueName: \"kubernetes.io/projected/f7863475-333d-4f6c-a720-aa59b89d99d2-kube-api-access-jxn4h\") pod \"f7863475-333d-4f6c-a720-aa59b89d99d2\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " Jul 09 10:25:22.825260 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.825235 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-tmp-dir\") pod \"f7863475-333d-4f6c-a720-aa59b89d99d2\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " Jul 09 10:25:22.825260 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.825261 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-home\") pod \"f7863475-333d-4f6c-a720-aa59b89d99d2\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " Jul 09 10:25:22.825596 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.825310 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-dshm\") pod \"f7863475-333d-4f6c-a720-aa59b89d99d2\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " Jul 09 10:25:22.825596 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.825326 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7863475-333d-4f6c-a720-aa59b89d99d2-tls-certs\") pod \"f7863475-333d-4f6c-a720-aa59b89d99d2\" (UID: \"f7863475-333d-4f6c-a720-aa59b89d99d2\") " Jul 09 10:25:22.825596 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.825520 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-model-cache" (OuterVolumeSpecName: "model-cache") pod "f7863475-333d-4f6c-a720-aa59b89d99d2" (UID: "f7863475-333d-4f6c-a720-aa59b89d99d2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:22.825596 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.825529 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f7863475-333d-4f6c-a720-aa59b89d99d2" (UID: "f7863475-333d-4f6c-a720-aa59b89d99d2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:22.825784 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.825658 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-home" (OuterVolumeSpecName: "home") pod "f7863475-333d-4f6c-a720-aa59b89d99d2" (UID: "f7863475-333d-4f6c-a720-aa59b89d99d2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:22.827518 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.827492 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-dshm" (OuterVolumeSpecName: "dshm") pod "f7863475-333d-4f6c-a720-aa59b89d99d2" (UID: "f7863475-333d-4f6c-a720-aa59b89d99d2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 10:25:22.827518 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.827506 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7863475-333d-4f6c-a720-aa59b89d99d2-kube-api-access-jxn4h" (OuterVolumeSpecName: "kube-api-access-jxn4h") pod "f7863475-333d-4f6c-a720-aa59b89d99d2" (UID: "f7863475-333d-4f6c-a720-aa59b89d99d2"). InnerVolumeSpecName "kube-api-access-jxn4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:25:22.827518 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.827507 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7863475-333d-4f6c-a720-aa59b89d99d2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f7863475-333d-4f6c-a720-aa59b89d99d2" (UID: "f7863475-333d-4f6c-a720-aa59b89d99d2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 10:25:22.926899 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.926861 2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-model-cache\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:22.926899 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.926892 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxn4h\" (UniqueName: \"kubernetes.io/projected/f7863475-333d-4f6c-a720-aa59b89d99d2-kube-api-access-jxn4h\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:22.926899 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.926903 2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-tmp-dir\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:22.927126 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.926912 2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-home\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:22.927126 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.926920 2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7863475-333d-4f6c-a720-aa59b89d99d2-dshm\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:22.927126 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:22.926927 2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7863475-333d-4f6c-a720-aa59b89d99d2-tls-certs\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:23.629564 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:23.629524 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" Jul 09 10:25:23.629564 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:23.629537 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn" event={"ID":"f7863475-333d-4f6c-a720-aa59b89d99d2","Type":"ContainerDied","Data":"bc1aa082204afda2a45bb5dda179925d25a447b788d37ee4564a814fd4e37158"} Jul 09 10:25:23.630054 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:23.629593 2582 scope.go:117] "RemoveContainer" containerID="fa020320221125d42f13e6e425ab414fa322d56b0b2b44510fe85de9c117207c" Jul 09 10:25:23.654780 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:23.654750 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn"] Jul 09 10:25:23.660010 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:23.659986 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-14869d/test-model-14869d-kserve-d5b7797-89wnn"] Jul 09 10:25:24.042299 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:24.042262 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de29a743-e1d9-4b87-b6ed-1dee9dc64134" path="/var/lib/kubelet/pods/de29a743-e1d9-4b87-b6ed-1dee9dc64134/volumes" Jul 09 10:25:24.042668 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:24.042654 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7863475-333d-4f6c-a720-aa59b89d99d2" path="/var/lib/kubelet/pods/f7863475-333d-4f6c-a720-aa59b89d99d2/volumes" Jul 09 10:25:52.441750 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.441720 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k"] Jul 09 10:25:52.442239 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.441993 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" podUID="574fd649-383f-4da9-8372-8cacf1559819" containerName="maas-api" containerID="cri-o://41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a" gracePeriod=30 Jul 09 10:25:52.680911 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.680880 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:25:52.751734 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.751701 2582 generic.go:356] "Generic (PLEG): container finished" podID="574fd649-383f-4da9-8372-8cacf1559819" containerID="41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a" exitCode=0 Jul 09 10:25:52.751894 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.751740 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" event={"ID":"574fd649-383f-4da9-8372-8cacf1559819","Type":"ContainerDied","Data":"41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a"} Jul 09 10:25:52.751894 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.751748 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" Jul 09 10:25:52.751894 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.751806 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k" event={"ID":"574fd649-383f-4da9-8372-8cacf1559819","Type":"ContainerDied","Data":"f628c77931808f07f5fdcff325e13751a44fb54f0348a18fb38e1e74e29cbf07"} Jul 09 10:25:52.751894 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.751828 2582 scope.go:117] "RemoveContainer" containerID="41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a" Jul 09 10:25:52.759646 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.759606 2582 scope.go:117] "RemoveContainer" containerID="41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a" Jul 09 10:25:52.759920 ip-10-0-134-220 kubenswrapper[2582]: E0709 10:25:52.759892 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a\": container with ID starting with 41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a not found: ID does not exist" containerID="41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a" Jul 09 10:25:52.760000 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.759919 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a"} err="failed to get container status \"41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a\": rpc error: code = NotFound desc = could not find container \"41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a\": container with ID starting with 41cf1fc28e77e9e651aae744563ac8761aa2511fc1a156a1e7b4298e3a615f6a not found: ID does not exist" Jul 09 10:25:52.771344 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.771327 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/574fd649-383f-4da9-8372-8cacf1559819-openshift-service-ca\") pod \"574fd649-383f-4da9-8372-8cacf1559819\" (UID: \"574fd649-383f-4da9-8372-8cacf1559819\") " Jul 09 10:25:52.771411 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.771363 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/574fd649-383f-4da9-8372-8cacf1559819-maas-api-tls\") pod \"574fd649-383f-4da9-8372-8cacf1559819\" (UID: \"574fd649-383f-4da9-8372-8cacf1559819\") " Jul 09 10:25:52.771470 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.771453 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wvhq\" (UniqueName: \"kubernetes.io/projected/574fd649-383f-4da9-8372-8cacf1559819-kube-api-access-5wvhq\") pod \"574fd649-383f-4da9-8372-8cacf1559819\" (UID: \"574fd649-383f-4da9-8372-8cacf1559819\") " Jul 09 10:25:52.771728 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.771704 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/574fd649-383f-4da9-8372-8cacf1559819-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "574fd649-383f-4da9-8372-8cacf1559819" (UID: "574fd649-383f-4da9-8372-8cacf1559819"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:25:52.773681 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.773660 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/574fd649-383f-4da9-8372-8cacf1559819-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "574fd649-383f-4da9-8372-8cacf1559819" (UID: "574fd649-383f-4da9-8372-8cacf1559819"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 10:25:52.773773 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.773678 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/574fd649-383f-4da9-8372-8cacf1559819-kube-api-access-5wvhq" (OuterVolumeSpecName: "kube-api-access-5wvhq") pod "574fd649-383f-4da9-8372-8cacf1559819" (UID: "574fd649-383f-4da9-8372-8cacf1559819"). InnerVolumeSpecName "kube-api-access-5wvhq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:25:52.872023 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.871992 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wvhq\" (UniqueName: \"kubernetes.io/projected/574fd649-383f-4da9-8372-8cacf1559819-kube-api-access-5wvhq\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:52.872023 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.872019 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/574fd649-383f-4da9-8372-8cacf1559819-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:52.872023 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:52.872030 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/574fd649-383f-4da9-8372-8cacf1559819-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:25:53.076072 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:53.076042 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k"] Jul 09 10:25:53.081389 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:53.081360 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-14869d-6d7f5d6495-86j8k"] Jul 09 10:25:54.040961 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:25:54.040923 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="574fd649-383f-4da9-8372-8cacf1559819" path="/var/lib/kubelet/pods/574fd649-383f-4da9-8372-8cacf1559819/volumes" Jul 09 10:26:03.553959 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.553921 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk"] Jul 09 10:26:03.554360 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.554344 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="574fd649-383f-4da9-8372-8cacf1559819" containerName="maas-api" Jul 09 10:26:03.554405 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.554362 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="574fd649-383f-4da9-8372-8cacf1559819" containerName="maas-api" Jul 09 10:26:03.554405 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.554379 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de29a743-e1d9-4b87-b6ed-1dee9dc64134" containerName="main" Jul 09 10:26:03.554405 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.554387 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="de29a743-e1d9-4b87-b6ed-1dee9dc64134" containerName="main" Jul 09 10:26:03.554405 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.554396 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7863475-333d-4f6c-a720-aa59b89d99d2" containerName="main" Jul 09 10:26:03.554405 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.554405 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7863475-333d-4f6c-a720-aa59b89d99d2" containerName="main" Jul 09 10:26:03.554557 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.554484 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="574fd649-383f-4da9-8372-8cacf1559819" containerName="maas-api" Jul 09 10:26:03.554557 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.554495 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7863475-333d-4f6c-a720-aa59b89d99d2" containerName="main" Jul 09 10:26:03.554557 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.554505 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="de29a743-e1d9-4b87-b6ed-1dee9dc64134" containerName="main" Jul 09 10:26:03.558935 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.558913 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:03.561951 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.561918 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 10:26:03.562111 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.562042 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-npsbz\"" Jul 09 10:26:03.563013 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.562994 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-73cf2e-serving-cert\"" Jul 09 10:26:03.563112 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.563001 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 10:26:03.570419 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.570396 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk"] Jul 09 10:26:03.665872 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.665833 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-maas-api-tls\") pod \"maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk\" (UID: \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:03.666069 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.665889 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-openshift-service-ca\") pod \"maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk\" (UID: \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:03.666069 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.665927 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-974kp\" (UniqueName: \"kubernetes.io/projected/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-kube-api-access-974kp\") pod \"maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk\" (UID: \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:03.766683 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.766654 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-maas-api-tls\") pod \"maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk\" (UID: \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:03.766871 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.766704 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-openshift-service-ca\") pod \"maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk\" (UID: \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:03.766871 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.766836 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-974kp\" (UniqueName: \"kubernetes.io/projected/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-kube-api-access-974kp\") pod \"maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk\" (UID: \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:03.767383 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.767363 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-openshift-service-ca\") pod \"maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk\" (UID: \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:03.769172 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.769150 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-maas-api-tls\") pod \"maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk\" (UID: \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:03.777078 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.777053 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-974kp\" (UniqueName: \"kubernetes.io/projected/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-kube-api-access-974kp\") pod \"maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk\" (UID: \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:03.870821 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:03.870732 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:04.005917 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:04.005886 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk"] Jul 09 10:26:04.008908 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:26:04.008882 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf4ebe19_e763_462e_adc2_cbccb1bfc0cc.slice/crio-fe1178e991ba4b495c7920fd06f757e668de86d297845aac7cb0205a5c7db708 WatchSource:0}: Error finding container fe1178e991ba4b495c7920fd06f757e668de86d297845aac7cb0205a5c7db708: Status 404 returned error can't find the container with id fe1178e991ba4b495c7920fd06f757e668de86d297845aac7cb0205a5c7db708 Jul 09 10:26:04.804982 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:04.804904 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" event={"ID":"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc","Type":"ContainerStarted","Data":"08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44"} Jul 09 10:26:04.804982 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:04.804958 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" event={"ID":"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc","Type":"ContainerStarted","Data":"fe1178e991ba4b495c7920fd06f757e668de86d297845aac7cb0205a5c7db708"} Jul 09 10:26:04.805403 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:04.805015 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:04.822579 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:04.822535 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" podStartSLOduration=1.334858305 podStartE2EDuration="1.822518669s" totalImagesPullingTime="487.660364ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:26:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:26:04.822044485 +0000 UTC m=+1912.080994343" watchObservedRunningTime="2026-07-09 10:26:04.822518669 +0000 UTC m=+1912.081468530" Jul 09 10:26:10.816382 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:10.816358 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:11.758131 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.758101 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2"] Jul 09 10:26:11.761647 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.761600 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:11.764053 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.764025 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-2b323e-serving-cert\"" Jul 09 10:26:11.774175 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.774148 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2"] Jul 09 10:26:11.832544 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.832511 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvrrv\" (UniqueName: \"kubernetes.io/projected/bcb09e5a-6256-469b-8d66-8ee7981f429d-kube-api-access-xvrrv\") pod \"maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2\" (UID: \"bcb09e5a-6256-469b-8d66-8ee7981f429d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:11.832949 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.832572 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb09e5a-6256-469b-8d66-8ee7981f429d-openshift-service-ca\") pod \"maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2\" (UID: \"bcb09e5a-6256-469b-8d66-8ee7981f429d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:11.832949 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.832637 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcb09e5a-6256-469b-8d66-8ee7981f429d-maas-api-tls\") pod \"maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2\" (UID: \"bcb09e5a-6256-469b-8d66-8ee7981f429d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:11.933353 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.933321 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb09e5a-6256-469b-8d66-8ee7981f429d-openshift-service-ca\") pod \"maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2\" (UID: \"bcb09e5a-6256-469b-8d66-8ee7981f429d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:11.933566 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.933372 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcb09e5a-6256-469b-8d66-8ee7981f429d-maas-api-tls\") pod \"maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2\" (UID: \"bcb09e5a-6256-469b-8d66-8ee7981f429d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:11.933566 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.933434 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvrrv\" (UniqueName: \"kubernetes.io/projected/bcb09e5a-6256-469b-8d66-8ee7981f429d-kube-api-access-xvrrv\") pod \"maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2\" (UID: \"bcb09e5a-6256-469b-8d66-8ee7981f429d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:11.934129 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.934107 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb09e5a-6256-469b-8d66-8ee7981f429d-openshift-service-ca\") pod \"maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2\" (UID: \"bcb09e5a-6256-469b-8d66-8ee7981f429d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:11.935829 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.935808 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcb09e5a-6256-469b-8d66-8ee7981f429d-maas-api-tls\") pod \"maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2\" (UID: \"bcb09e5a-6256-469b-8d66-8ee7981f429d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:11.943778 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:11.943755 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvrrv\" (UniqueName: \"kubernetes.io/projected/bcb09e5a-6256-469b-8d66-8ee7981f429d-kube-api-access-xvrrv\") pod \"maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2\" (UID: \"bcb09e5a-6256-469b-8d66-8ee7981f429d\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:12.072969 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:12.072871 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:12.205357 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:12.205329 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2"] Jul 09 10:26:12.207870 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:26:12.207842 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb09e5a_6256_469b_8d66_8ee7981f429d.slice/crio-8a359d0c4b80bc4c5ad43570aa4c66705e12b0701729514f69956a4e571f61c8 WatchSource:0}: Error finding container 8a359d0c4b80bc4c5ad43570aa4c66705e12b0701729514f69956a4e571f61c8: Status 404 returned error can't find the container with id 8a359d0c4b80bc4c5ad43570aa4c66705e12b0701729514f69956a4e571f61c8 Jul 09 10:26:12.839224 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:12.839187 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" event={"ID":"bcb09e5a-6256-469b-8d66-8ee7981f429d","Type":"ContainerStarted","Data":"4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0"} Jul 09 10:26:12.839703 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:12.839242 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" event={"ID":"bcb09e5a-6256-469b-8d66-8ee7981f429d","Type":"ContainerStarted","Data":"8a359d0c4b80bc4c5ad43570aa4c66705e12b0701729514f69956a4e571f61c8"} Jul 09 10:26:12.839703 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:12.839272 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:12.858665 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:12.858580 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" podStartSLOduration=1.451248554 podStartE2EDuration="1.858562793s" totalImagesPullingTime="407.314239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:26:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:26:12.857878924 +0000 UTC m=+1920.116828783" watchObservedRunningTime="2026-07-09 10:26:12.858562793 +0000 UTC m=+1920.117512652" Jul 09 10:26:18.849430 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:18.849391 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:25.481880 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.481845 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk"] Jul 09 10:26:25.482292 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.482098 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" podUID="cf4ebe19-e763-462e-adc2-cbccb1bfc0cc" containerName="maas-api" containerID="cri-o://08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44" gracePeriod=30 Jul 09 10:26:25.717934 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.717909 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:25.848520 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.848433 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-maas-api-tls\") pod \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\" (UID: \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\") " Jul 09 10:26:25.848520 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.848473 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-openshift-service-ca\") pod \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\" (UID: \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\") " Jul 09 10:26:25.848723 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.848562 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-974kp\" (UniqueName: \"kubernetes.io/projected/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-kube-api-access-974kp\") pod \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\" (UID: \"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc\") " Jul 09 10:26:25.848929 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.848904 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cf4ebe19-e763-462e-adc2-cbccb1bfc0cc" (UID: "cf4ebe19-e763-462e-adc2-cbccb1bfc0cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:26:25.850663 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.850634 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-kube-api-access-974kp" (OuterVolumeSpecName: "kube-api-access-974kp") pod "cf4ebe19-e763-462e-adc2-cbccb1bfc0cc" (UID: "cf4ebe19-e763-462e-adc2-cbccb1bfc0cc"). InnerVolumeSpecName "kube-api-access-974kp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:26:25.850663 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.850646 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cf4ebe19-e763-462e-adc2-cbccb1bfc0cc" (UID: "cf4ebe19-e763-462e-adc2-cbccb1bfc0cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 10:26:25.899372 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.899342 2582 generic.go:356] "Generic (PLEG): container finished" podID="cf4ebe19-e763-462e-adc2-cbccb1bfc0cc" containerID="08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44" exitCode=0 Jul 09 10:26:25.899497 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.899391 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" Jul 09 10:26:25.899497 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.899435 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" event={"ID":"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc","Type":"ContainerDied","Data":"08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44"} Jul 09 10:26:25.899497 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.899489 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk" event={"ID":"cf4ebe19-e763-462e-adc2-cbccb1bfc0cc","Type":"ContainerDied","Data":"fe1178e991ba4b495c7920fd06f757e668de86d297845aac7cb0205a5c7db708"} Jul 09 10:26:25.899598 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.899511 2582 scope.go:117] "RemoveContainer" containerID="08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44" Jul 09 10:26:25.908241 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.908218 2582 scope.go:117] "RemoveContainer" containerID="08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44" Jul 09 10:26:25.908481 ip-10-0-134-220 kubenswrapper[2582]: E0709 10:26:25.908463 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44\": container with ID starting with 08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44 not found: ID does not exist" containerID="08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44" Jul 09 10:26:25.908533 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.908496 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44"} err="failed to get container status \"08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44\": rpc error: code = NotFound desc = could not find container \"08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44\": container with ID starting with 08083d70d2bfcd5e1814134df817524b6c5bddcb35d12f461a67df14dc3a1f44 not found: ID does not exist" Jul 09 10:26:25.929397 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.929375 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk"] Jul 09 10:26:25.945645 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.945624 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-73cf2e-57c955dd4f-xsxkk"] Jul 09 10:26:25.949439 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.949420 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:26:25.949489 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.949442 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:26:25.949489 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:25.949452 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-974kp\" (UniqueName: \"kubernetes.io/projected/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc-kube-api-access-974kp\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:26:26.040050 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:26.040021 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf4ebe19-e763-462e-adc2-cbccb1bfc0cc" path="/var/lib/kubelet/pods/cf4ebe19-e763-462e-adc2-cbccb1bfc0cc/volumes" Jul 09 10:26:33.546433 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.546401 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2"] Jul 09 10:26:33.546859 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.546644 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" podUID="bcb09e5a-6256-469b-8d66-8ee7981f429d" containerName="maas-api" containerID="cri-o://4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0" gracePeriod=30 Jul 09 10:26:33.792099 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.792072 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:33.914595 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.914503 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb09e5a-6256-469b-8d66-8ee7981f429d-openshift-service-ca\") pod \"bcb09e5a-6256-469b-8d66-8ee7981f429d\" (UID: \"bcb09e5a-6256-469b-8d66-8ee7981f429d\") " Jul 09 10:26:33.914595 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.914585 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvrrv\" (UniqueName: \"kubernetes.io/projected/bcb09e5a-6256-469b-8d66-8ee7981f429d-kube-api-access-xvrrv\") pod \"bcb09e5a-6256-469b-8d66-8ee7981f429d\" (UID: \"bcb09e5a-6256-469b-8d66-8ee7981f429d\") " Jul 09 10:26:33.914794 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.914639 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcb09e5a-6256-469b-8d66-8ee7981f429d-maas-api-tls\") pod \"bcb09e5a-6256-469b-8d66-8ee7981f429d\" (UID: \"bcb09e5a-6256-469b-8d66-8ee7981f429d\") " Jul 09 10:26:33.915018 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.914974 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb09e5a-6256-469b-8d66-8ee7981f429d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bcb09e5a-6256-469b-8d66-8ee7981f429d" (UID: "bcb09e5a-6256-469b-8d66-8ee7981f429d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:26:33.916784 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.916759 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb09e5a-6256-469b-8d66-8ee7981f429d-kube-api-access-xvrrv" (OuterVolumeSpecName: "kube-api-access-xvrrv") pod "bcb09e5a-6256-469b-8d66-8ee7981f429d" (UID: "bcb09e5a-6256-469b-8d66-8ee7981f429d"). InnerVolumeSpecName "kube-api-access-xvrrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:26:33.916878 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.916812 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcb09e5a-6256-469b-8d66-8ee7981f429d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bcb09e5a-6256-469b-8d66-8ee7981f429d" (UID: "bcb09e5a-6256-469b-8d66-8ee7981f429d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 10:26:33.932188 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.932159 2582 generic.go:356] "Generic (PLEG): container finished" podID="bcb09e5a-6256-469b-8d66-8ee7981f429d" containerID="4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0" exitCode=0 Jul 09 10:26:33.932309 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.932202 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" Jul 09 10:26:33.932309 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.932229 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" event={"ID":"bcb09e5a-6256-469b-8d66-8ee7981f429d","Type":"ContainerDied","Data":"4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0"} Jul 09 10:26:33.932309 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.932270 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2" event={"ID":"bcb09e5a-6256-469b-8d66-8ee7981f429d","Type":"ContainerDied","Data":"8a359d0c4b80bc4c5ad43570aa4c66705e12b0701729514f69956a4e571f61c8"} Jul 09 10:26:33.932309 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.932291 2582 scope.go:117] "RemoveContainer" containerID="4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0" Jul 09 10:26:33.940797 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.940782 2582 scope.go:117] "RemoveContainer" containerID="4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0" Jul 09 10:26:33.941024 ip-10-0-134-220 kubenswrapper[2582]: E0709 10:26:33.941009 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0\": container with ID starting with 4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0 not found: ID does not exist" containerID="4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0" Jul 09 10:26:33.941074 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.941031 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0"} err="failed to get container status \"4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0\": rpc error: code = NotFound desc = could not find container \"4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0\": container with ID starting with 4f60efd19ae4c6bc814df618aaba0dbc8ef4d205f1b6566e7f2402bbdb0ba3a0 not found: ID does not exist" Jul 09 10:26:33.955705 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.955679 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2"] Jul 09 10:26:33.959827 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:33.959806 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2b323e-7d4fdc4785-wlxj2"] Jul 09 10:26:34.015964 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:34.015928 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvrrv\" (UniqueName: \"kubernetes.io/projected/bcb09e5a-6256-469b-8d66-8ee7981f429d-kube-api-access-xvrrv\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:26:34.015964 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:34.015958 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bcb09e5a-6256-469b-8d66-8ee7981f429d-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:26:34.015964 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:34.015970 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bcb09e5a-6256-469b-8d66-8ee7981f429d-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:26:34.041493 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:34.041458 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcb09e5a-6256-469b-8d66-8ee7981f429d" path="/var/lib/kubelet/pods/bcb09e5a-6256-469b-8d66-8ee7981f429d/volumes" Jul 09 10:26:43.896352 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.896320 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr"] Jul 09 10:26:43.896796 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.896762 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf4ebe19-e763-462e-adc2-cbccb1bfc0cc" containerName="maas-api" Jul 09 10:26:43.896796 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.896778 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf4ebe19-e763-462e-adc2-cbccb1bfc0cc" containerName="maas-api" Jul 09 10:26:43.896874 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.896799 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb09e5a-6256-469b-8d66-8ee7981f429d" containerName="maas-api" Jul 09 10:26:43.896874 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.896808 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb09e5a-6256-469b-8d66-8ee7981f429d" containerName="maas-api" Jul 09 10:26:43.896946 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.896879 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf4ebe19-e763-462e-adc2-cbccb1bfc0cc" containerName="maas-api" Jul 09 10:26:43.896946 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.896893 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcb09e5a-6256-469b-8d66-8ee7981f429d" containerName="maas-api" Jul 09 10:26:43.899856 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.899838 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:43.907428 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.907397 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-b5586e-serving-cert\"" Jul 09 10:26:43.907639 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.907436 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 09 10:26:43.907639 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.907576 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 09 10:26:43.908560 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.908537 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-npsbz\"" Jul 09 10:26:43.912264 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.912241 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr"] Jul 09 10:26:43.994090 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.994052 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cad9a2e-916b-4bce-b54a-04351b1086b0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr\" (UID: \"9cad9a2e-916b-4bce-b54a-04351b1086b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:43.994271 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.994100 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cad9a2e-916b-4bce-b54a-04351b1086b0-maas-api-tls\") pod \"maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr\" (UID: \"9cad9a2e-916b-4bce-b54a-04351b1086b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:43.994271 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:43.994119 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr7lj\" (UniqueName: \"kubernetes.io/projected/9cad9a2e-916b-4bce-b54a-04351b1086b0-kube-api-access-sr7lj\") pod \"maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr\" (UID: \"9cad9a2e-916b-4bce-b54a-04351b1086b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:44.094840 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:44.094808 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cad9a2e-916b-4bce-b54a-04351b1086b0-maas-api-tls\") pod \"maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr\" (UID: \"9cad9a2e-916b-4bce-b54a-04351b1086b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:44.094840 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:44.094842 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr7lj\" (UniqueName: \"kubernetes.io/projected/9cad9a2e-916b-4bce-b54a-04351b1086b0-kube-api-access-sr7lj\") pod \"maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr\" (UID: \"9cad9a2e-916b-4bce-b54a-04351b1086b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:44.095145 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:44.095117 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cad9a2e-916b-4bce-b54a-04351b1086b0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr\" (UID: \"9cad9a2e-916b-4bce-b54a-04351b1086b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:44.095819 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:44.095797 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cad9a2e-916b-4bce-b54a-04351b1086b0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr\" (UID: \"9cad9a2e-916b-4bce-b54a-04351b1086b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:44.097299 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:44.097276 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cad9a2e-916b-4bce-b54a-04351b1086b0-maas-api-tls\") pod \"maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr\" (UID: \"9cad9a2e-916b-4bce-b54a-04351b1086b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:44.110116 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:44.110095 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr7lj\" (UniqueName: \"kubernetes.io/projected/9cad9a2e-916b-4bce-b54a-04351b1086b0-kube-api-access-sr7lj\") pod \"maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr\" (UID: \"9cad9a2e-916b-4bce-b54a-04351b1086b0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:44.219014 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:44.218912 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:44.354432 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:44.354274 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr"] Jul 09 10:26:44.357058 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:26:44.357031 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cad9a2e_916b_4bce_b54a_04351b1086b0.slice/crio-26b5d3c071bfebb903c341d51baf53aa35d17ff1dabb41154e709235c3b7740d WatchSource:0}: Error finding container 26b5d3c071bfebb903c341d51baf53aa35d17ff1dabb41154e709235c3b7740d: Status 404 returned error can't find the container with id 26b5d3c071bfebb903c341d51baf53aa35d17ff1dabb41154e709235c3b7740d Jul 09 10:26:44.978806 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:44.978776 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" event={"ID":"9cad9a2e-916b-4bce-b54a-04351b1086b0","Type":"ContainerStarted","Data":"5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b"} Jul 09 10:26:44.979152 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:44.978823 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" event={"ID":"9cad9a2e-916b-4bce-b54a-04351b1086b0","Type":"ContainerStarted","Data":"26b5d3c071bfebb903c341d51baf53aa35d17ff1dabb41154e709235c3b7740d"} Jul 09 10:26:44.979152 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:44.978865 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:45.000557 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:45.000511 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" podStartSLOduration=1.5586681489999998 podStartE2EDuration="2.000495702s" totalImagesPullingTime="441.827553ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:26:44.998836135 +0000 UTC m=+1952.257785995" watchObservedRunningTime="2026-07-09 10:26:45.000495702 +0000 UTC m=+1952.259445561" Jul 09 10:26:50.988296 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:50.988264 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:26:51.881814 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:51.881777 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2"] Jul 09 10:26:51.885467 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:51.885445 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:26:51.887883 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:51.887862 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-a0b44e-serving-cert\"" Jul 09 10:26:51.896389 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:51.896362 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2"] Jul 09 10:26:51.962919 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:51.962889 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93ff43da-0240-42ca-869c-e9aa5086d1e0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2\" (UID: \"93ff43da-0240-42ca-869c-e9aa5086d1e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:26:51.963097 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:51.962933 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93ff43da-0240-42ca-869c-e9aa5086d1e0-maas-api-tls\") pod \"maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2\" (UID: \"93ff43da-0240-42ca-869c-e9aa5086d1e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:26:51.963097 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:51.962986 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm2mv\" (UniqueName: \"kubernetes.io/projected/93ff43da-0240-42ca-869c-e9aa5086d1e0-kube-api-access-pm2mv\") pod \"maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2\" (UID: \"93ff43da-0240-42ca-869c-e9aa5086d1e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:26:52.064294 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:52.064259 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93ff43da-0240-42ca-869c-e9aa5086d1e0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2\" (UID: \"93ff43da-0240-42ca-869c-e9aa5086d1e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:26:52.064689 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:52.064305 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93ff43da-0240-42ca-869c-e9aa5086d1e0-maas-api-tls\") pod \"maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2\" (UID: \"93ff43da-0240-42ca-869c-e9aa5086d1e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:26:52.064689 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:52.064325 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm2mv\" (UniqueName: \"kubernetes.io/projected/93ff43da-0240-42ca-869c-e9aa5086d1e0-kube-api-access-pm2mv\") pod \"maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2\" (UID: \"93ff43da-0240-42ca-869c-e9aa5086d1e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:26:52.065041 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:52.065011 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93ff43da-0240-42ca-869c-e9aa5086d1e0-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2\" (UID: \"93ff43da-0240-42ca-869c-e9aa5086d1e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:26:52.066788 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:52.066760 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93ff43da-0240-42ca-869c-e9aa5086d1e0-maas-api-tls\") pod \"maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2\" (UID: \"93ff43da-0240-42ca-869c-e9aa5086d1e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:26:52.073963 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:52.073940 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm2mv\" (UniqueName: \"kubernetes.io/projected/93ff43da-0240-42ca-869c-e9aa5086d1e0-kube-api-access-pm2mv\") pod \"maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2\" (UID: \"93ff43da-0240-42ca-869c-e9aa5086d1e0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:26:52.197173 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:52.197087 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:26:52.331433 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:52.331409 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2"] Jul 09 10:26:52.334187 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:26:52.334158 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93ff43da_0240_42ca_869c_e9aa5086d1e0.slice/crio-f35a81497b06ce8ed6a7aa9987185005e7ce4de381a77ee25abbff56c7559377 WatchSource:0}: Error finding container f35a81497b06ce8ed6a7aa9987185005e7ce4de381a77ee25abbff56c7559377: Status 404 returned error can't find the container with id f35a81497b06ce8ed6a7aa9987185005e7ce4de381a77ee25abbff56c7559377 Jul 09 10:26:53.013708 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:53.013674 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" event={"ID":"93ff43da-0240-42ca-869c-e9aa5086d1e0","Type":"ContainerStarted","Data":"6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d"} Jul 09 10:26:53.013874 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:53.013769 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" event={"ID":"93ff43da-0240-42ca-869c-e9aa5086d1e0","Type":"ContainerStarted","Data":"f35a81497b06ce8ed6a7aa9987185005e7ce4de381a77ee25abbff56c7559377"} Jul 09 10:26:53.013874 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:53.013807 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:26:53.034472 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:53.034420 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" podStartSLOduration=1.650312193 podStartE2EDuration="2.034403827s" totalImagesPullingTime="384.091634ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 10:26:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:26:53.032729347 +0000 UTC m=+1960.291679207" watchObservedRunningTime="2026-07-09 10:26:53.034403827 +0000 UTC m=+1960.293353686" Jul 09 10:26:59.023709 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:26:59.023673 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:30:02.749579 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:02.749537 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr"] Jul 09 10:30:02.750051 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:02.749824 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" podUID="9cad9a2e-916b-4bce-b54a-04351b1086b0" containerName="maas-api" containerID="cri-o://5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b" gracePeriod=30 Jul 09 10:30:03.001267 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.001205 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:30:03.138232 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.138197 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cad9a2e-916b-4bce-b54a-04351b1086b0-maas-api-tls\") pod \"9cad9a2e-916b-4bce-b54a-04351b1086b0\" (UID: \"9cad9a2e-916b-4bce-b54a-04351b1086b0\") " Jul 09 10:30:03.138417 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.138261 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cad9a2e-916b-4bce-b54a-04351b1086b0-openshift-service-ca\") pod \"9cad9a2e-916b-4bce-b54a-04351b1086b0\" (UID: \"9cad9a2e-916b-4bce-b54a-04351b1086b0\") " Jul 09 10:30:03.138417 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.138308 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr7lj\" (UniqueName: \"kubernetes.io/projected/9cad9a2e-916b-4bce-b54a-04351b1086b0-kube-api-access-sr7lj\") pod \"9cad9a2e-916b-4bce-b54a-04351b1086b0\" (UID: \"9cad9a2e-916b-4bce-b54a-04351b1086b0\") " Jul 09 10:30:03.138725 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.138695 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cad9a2e-916b-4bce-b54a-04351b1086b0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9cad9a2e-916b-4bce-b54a-04351b1086b0" (UID: "9cad9a2e-916b-4bce-b54a-04351b1086b0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:30:03.140355 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.140332 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cad9a2e-916b-4bce-b54a-04351b1086b0-kube-api-access-sr7lj" (OuterVolumeSpecName: "kube-api-access-sr7lj") pod "9cad9a2e-916b-4bce-b54a-04351b1086b0" (UID: "9cad9a2e-916b-4bce-b54a-04351b1086b0"). InnerVolumeSpecName "kube-api-access-sr7lj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:30:03.140444 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.140357 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cad9a2e-916b-4bce-b54a-04351b1086b0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9cad9a2e-916b-4bce-b54a-04351b1086b0" (UID: "9cad9a2e-916b-4bce-b54a-04351b1086b0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 10:30:03.239094 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.239054 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9cad9a2e-916b-4bce-b54a-04351b1086b0-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:30:03.239094 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.239089 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9cad9a2e-916b-4bce-b54a-04351b1086b0-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:30:03.239094 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.239098 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr7lj\" (UniqueName: \"kubernetes.io/projected/9cad9a2e-916b-4bce-b54a-04351b1086b0-kube-api-access-sr7lj\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:30:03.816734 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.816693 2582 generic.go:356] "Generic (PLEG): container finished" podID="9cad9a2e-916b-4bce-b54a-04351b1086b0" containerID="5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b" exitCode=0 Jul 09 10:30:03.817151 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.816755 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" Jul 09 10:30:03.817151 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.816778 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" event={"ID":"9cad9a2e-916b-4bce-b54a-04351b1086b0","Type":"ContainerDied","Data":"5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b"} Jul 09 10:30:03.817151 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.816837 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr" event={"ID":"9cad9a2e-916b-4bce-b54a-04351b1086b0","Type":"ContainerDied","Data":"26b5d3c071bfebb903c341d51baf53aa35d17ff1dabb41154e709235c3b7740d"} Jul 09 10:30:03.817151 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.816860 2582 scope.go:117] "RemoveContainer" containerID="5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b" Jul 09 10:30:03.825439 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.825422 2582 scope.go:117] "RemoveContainer" containerID="5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b" Jul 09 10:30:03.825739 ip-10-0-134-220 kubenswrapper[2582]: E0709 10:30:03.825721 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b\": container with ID starting with 5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b not found: ID does not exist" containerID="5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b" Jul 09 10:30:03.825810 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.825749 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b"} err="failed to get container status \"5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b\": rpc error: code = NotFound desc = could not find container \"5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b\": container with ID starting with 5155cf3078e9948bd3ee110deee44bd1201c519f77f6c9394caa571c2b80016b not found: ID does not exist" Jul 09 10:30:03.841153 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.841126 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr"] Jul 09 10:30:03.848924 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:03.848895 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b5586e-84649759f7-9vxnr"] Jul 09 10:30:04.043913 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:04.043880 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cad9a2e-916b-4bce-b54a-04351b1086b0" path="/var/lib/kubelet/pods/9cad9a2e-916b-4bce-b54a-04351b1086b0/volumes" Jul 09 10:30:10.002847 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.002766 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2"] Jul 09 10:30:10.003225 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.002969 2582 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" podUID="93ff43da-0240-42ca-869c-e9aa5086d1e0" containerName="maas-api" containerID="cri-o://6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d" gracePeriod=30 Jul 09 10:30:10.244896 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.244868 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:30:10.397803 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.397705 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pm2mv\" (UniqueName: \"kubernetes.io/projected/93ff43da-0240-42ca-869c-e9aa5086d1e0-kube-api-access-pm2mv\") pod \"93ff43da-0240-42ca-869c-e9aa5086d1e0\" (UID: \"93ff43da-0240-42ca-869c-e9aa5086d1e0\") " Jul 09 10:30:10.397803 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.397768 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93ff43da-0240-42ca-869c-e9aa5086d1e0-maas-api-tls\") pod \"93ff43da-0240-42ca-869c-e9aa5086d1e0\" (UID: \"93ff43da-0240-42ca-869c-e9aa5086d1e0\") " Jul 09 10:30:10.398033 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.397816 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93ff43da-0240-42ca-869c-e9aa5086d1e0-openshift-service-ca\") pod \"93ff43da-0240-42ca-869c-e9aa5086d1e0\" (UID: \"93ff43da-0240-42ca-869c-e9aa5086d1e0\") " Jul 09 10:30:10.398223 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.398194 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93ff43da-0240-42ca-869c-e9aa5086d1e0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "93ff43da-0240-42ca-869c-e9aa5086d1e0" (UID: "93ff43da-0240-42ca-869c-e9aa5086d1e0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 10:30:10.399876 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.399857 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93ff43da-0240-42ca-869c-e9aa5086d1e0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "93ff43da-0240-42ca-869c-e9aa5086d1e0" (UID: "93ff43da-0240-42ca-869c-e9aa5086d1e0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 10:30:10.400063 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.400037 2582 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93ff43da-0240-42ca-869c-e9aa5086d1e0-kube-api-access-pm2mv" (OuterVolumeSpecName: "kube-api-access-pm2mv") pod "93ff43da-0240-42ca-869c-e9aa5086d1e0" (UID: "93ff43da-0240-42ca-869c-e9aa5086d1e0"). InnerVolumeSpecName "kube-api-access-pm2mv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 10:30:10.498385 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.498341 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm2mv\" (UniqueName: \"kubernetes.io/projected/93ff43da-0240-42ca-869c-e9aa5086d1e0-kube-api-access-pm2mv\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:30:10.498385 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.498376 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/93ff43da-0240-42ca-869c-e9aa5086d1e0-maas-api-tls\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:30:10.498385 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.498385 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/93ff43da-0240-42ca-869c-e9aa5086d1e0-openshift-service-ca\") on node \"ip-10-0-134-220.ec2.internal\" DevicePath \"\"" Jul 09 10:30:10.849050 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.849011 2582 generic.go:356] "Generic (PLEG): container finished" podID="93ff43da-0240-42ca-869c-e9aa5086d1e0" containerID="6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d" exitCode=0 Jul 09 10:30:10.849226 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.849060 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" Jul 09 10:30:10.849226 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.849071 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" event={"ID":"93ff43da-0240-42ca-869c-e9aa5086d1e0","Type":"ContainerDied","Data":"6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d"} Jul 09 10:30:10.849226 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.849123 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2" event={"ID":"93ff43da-0240-42ca-869c-e9aa5086d1e0","Type":"ContainerDied","Data":"f35a81497b06ce8ed6a7aa9987185005e7ce4de381a77ee25abbff56c7559377"} Jul 09 10:30:10.849226 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.849154 2582 scope.go:117] "RemoveContainer" containerID="6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d" Jul 09 10:30:10.857920 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.857892 2582 scope.go:117] "RemoveContainer" containerID="6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d" Jul 09 10:30:10.858179 ip-10-0-134-220 kubenswrapper[2582]: E0709 10:30:10.858157 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d\": container with ID starting with 6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d not found: ID does not exist" containerID="6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d" Jul 09 10:30:10.858223 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.858187 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d"} err="failed to get container status \"6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d\": rpc error: code = NotFound desc = could not find container \"6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d\": container with ID starting with 6bf2251ab0c9abc748839cdc1b6fd4415564cddb275e27dae547a4d75567a09d not found: ID does not exist" Jul 09 10:30:10.873720 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.873689 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2"] Jul 09 10:30:10.877897 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:10.877875 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a0b44e-bc47dcff-m4ks2"] Jul 09 10:30:12.040299 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:12.040265 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93ff43da-0240-42ca-869c-e9aa5086d1e0" path="/var/lib/kubelet/pods/93ff43da-0240-42ca-869c-e9aa5086d1e0/volumes" Jul 09 10:30:17.431155 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:17.431126 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8588b87745-sjxt9_c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5/authorino/0.log" Jul 09 10:30:22.761499 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:22.761469 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5pk4g_04beab64-df60-4159-bfb8-07abe4f1c3b4/server/0.log" Jul 09 10:30:23.153255 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:23.153164 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cb598dddb-g6hv9_ed024081-9c20-476f-9e32-dd81e3cb50ea/manager/0.log" Jul 09 10:30:23.161846 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:23.161816 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cb598dddb-g6hv9_ed024081-9c20-476f-9e32-dd81e3cb50ea/copy-manifests/0.log" Jul 09 10:30:24.747777 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:24.747747 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8588b87745-sjxt9_c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5/authorino/0.log" Jul 09 10:30:25.108841 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:25.108761 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-c7sc5_78daf914-2660-4943-8122-d501ee70fc07/kuadrant-console-plugin/0.log" Jul 09 10:30:26.236818 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:26.236788 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589b555f76-xkg8s_c62438ba-e90e-4dc9-93f9-69115eab1e43/kube-auth-proxy/0.log" Jul 09 10:30:27.373850 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:27.373805 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-4njhm_2307e6f2-49fc-4055-a874-100bb8757ee9/main/0.log" Jul 09 10:30:27.624049 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:27.623968 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbfbsjs_87d9d933-22f4-4c09-95bd-cb4908653c49/main/0.log" Jul 09 10:30:27.871607 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:27.871569 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-f6snm_21771a3d-e377-42dd-91d3-5a99b6bc2c6a/main/0.log" Jul 09 10:30:36.007338 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:36.007303 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f987x_e0bd29c0-1d95-4097-9628-94da4639bcd5/global-pull-secret-syncer/0.log" Jul 09 10:30:36.227847 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:36.227818 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v97nr_5e95a1c6-e119-456a-91d0-7887e896e8ae/konnectivity-agent/0.log" Jul 09 10:30:36.254073 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:36.254041 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-220.ec2.internal_4780b8f2df1828ad4bb2a95b7e56dc50/haproxy/0.log" Jul 09 10:30:42.152093 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:42.152053 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8588b87745-sjxt9_c94bc8d0-ab3e-4a2c-9a72-c4cde612dfb5/authorino/0.log" Jul 09 10:30:42.297181 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:42.297151 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-c7sc5_78daf914-2660-4943-8122-d501ee70fc07/kuadrant-console-plugin/0.log" Jul 09 10:30:45.054187 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:45.054158 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m64qn_b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d/node-exporter/0.log" Jul 09 10:30:45.086429 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:45.086398 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m64qn_b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d/kube-rbac-proxy/0.log" Jul 09 10:30:45.123908 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:45.123881 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m64qn_b73ee8ba-d3aa-4669-a3bf-c5b1261cfe0d/init-textfile/0.log" Jul 09 10:30:46.296383 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.296346 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc"] Jul 09 10:30:46.296787 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.296648 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cad9a2e-916b-4bce-b54a-04351b1086b0" containerName="maas-api" Jul 09 10:30:46.296787 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.296660 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cad9a2e-916b-4bce-b54a-04351b1086b0" containerName="maas-api" Jul 09 10:30:46.296787 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.296680 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93ff43da-0240-42ca-869c-e9aa5086d1e0" containerName="maas-api" Jul 09 10:30:46.296787 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.296686 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="93ff43da-0240-42ca-869c-e9aa5086d1e0" containerName="maas-api" Jul 09 10:30:46.296787 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.296730 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="93ff43da-0240-42ca-869c-e9aa5086d1e0" containerName="maas-api" Jul 09 10:30:46.296787 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.296739 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cad9a2e-916b-4bce-b54a-04351b1086b0" containerName="maas-api" Jul 09 10:30:46.299600 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.299580 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.302499 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.302484 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qjnrh\"/\"default-dockercfg-q7c7q\"" Jul 09 10:30:46.302589 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.302508 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qjnrh\"/\"openshift-service-ca.crt\"" Jul 09 10:30:46.303659 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.303641 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qjnrh\"/\"kube-root-ca.crt\"" Jul 09 10:30:46.316869 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.316841 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc"] Jul 09 10:30:46.374171 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.374137 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-lib-modules\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.374368 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.374196 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-sys\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.374368 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.374226 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-podres\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.374368 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.374306 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-proc\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.374368 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.374356 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htw6f\" (UniqueName: \"kubernetes.io/projected/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-kube-api-access-htw6f\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.475480 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.475435 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-lib-modules\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.475711 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.475502 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-sys\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.475711 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.475530 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-podres\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.475711 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.475579 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-sys\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.475711 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.475587 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-proc\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.475711 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.475639 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-lib-modules\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.475711 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.475659 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-proc\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.475711 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.475701 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htw6f\" (UniqueName: \"kubernetes.io/projected/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-kube-api-access-htw6f\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.476042 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.475711 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-podres\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.485602 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.485558 2582 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htw6f\" (UniqueName: \"kubernetes.io/projected/e3be7ceb-0cc2-4b89-a998-55b2c29ed333-kube-api-access-htw6f\") pod \"perf-node-gather-daemonset-6wxxc\" (UID: \"e3be7ceb-0cc2-4b89-a998-55b2c29ed333\") " pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.609161 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.609070 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:46.739319 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:46.739296 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc"] Jul 09 10:30:46.741399 ip-10-0-134-220 kubenswrapper[2582]: W0709 10:30:46.741374 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode3be7ceb_0cc2_4b89_a998_55b2c29ed333.slice/crio-846d40f3a30cb6cac7e8f5a3652e951b7615fd1893509082bad8548b44a81136 WatchSource:0}: Error finding container 846d40f3a30cb6cac7e8f5a3652e951b7615fd1893509082bad8548b44a81136: Status 404 returned error can't find the container with id 846d40f3a30cb6cac7e8f5a3652e951b7615fd1893509082bad8548b44a81136 Jul 09 10:30:47.005788 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:47.005756 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" event={"ID":"e3be7ceb-0cc2-4b89-a998-55b2c29ed333","Type":"ContainerStarted","Data":"946cba30f2de3a483e6d8381385c11d96d978098933d53762610480a8fbda771"} Jul 09 10:30:47.005983 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:47.005815 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" event={"ID":"e3be7ceb-0cc2-4b89-a998-55b2c29ed333","Type":"ContainerStarted","Data":"846d40f3a30cb6cac7e8f5a3652e951b7615fd1893509082bad8548b44a81136"} Jul 09 10:30:47.005983 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:47.005843 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:47.035258 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:47.035203 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" podStartSLOduration=1.035186778 podStartE2EDuration="1.035186778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 10:30:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 10:30:47.03477642 +0000 UTC m=+2194.293726291" watchObservedRunningTime="2026-07-09 10:30:47.035186778 +0000 UTC m=+2194.294136637" Jul 09 10:30:47.175946 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:47.175908 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-6ndl2_58cfca84-a233-4ff9-b44f-1142e072a68b/networking-console-plugin/0.log" Jul 09 10:30:50.063541 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:50.063514 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6znj8_d1a11a4f-b946-4d7b-8540-d56dc533c8f0/dns/0.log" Jul 09 10:30:50.093913 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:50.093886 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6znj8_d1a11a4f-b946-4d7b-8540-d56dc533c8f0/kube-rbac-proxy/0.log" Jul 09 10:30:50.261181 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:50.261146 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-chwp7_8edce7b1-4310-4adf-b754-3169b53b9a90/dns-node-resolver/0.log" Jul 09 10:30:51.017475 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:51.017422 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74df7fc877-768cf_38e5e107-6837-4790-a6ba-1f9d2c44c725/registry/0.log" Jul 09 10:30:51.082279 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:51.082241 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-grjk4_8017af28-6540-46c3-9bc3-8b4ab3b83303/node-ca/0.log" Jul 09 10:30:52.313734 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:52.313705 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589b555f76-xkg8s_c62438ba-e90e-4dc9-93f9-69115eab1e43/kube-auth-proxy/0.log" Jul 09 10:30:53.019827 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:53.019799 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qjnrh/perf-node-gather-daemonset-6wxxc" Jul 09 10:30:53.418284 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:53.418209 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n2dcz_a502f840-af4d-417d-9113-35feb0400984/serve-healthcheck-canary/0.log" Jul 09 10:30:54.221192 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:54.221143 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ngzg4_8b9d4efd-4145-49fd-a355-042b72301086/kube-rbac-proxy/0.log" Jul 09 10:30:54.253722 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:54.253685 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ngzg4_8b9d4efd-4145-49fd-a355-042b72301086/exporter/0.log" Jul 09 10:30:54.286390 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:54.286364 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ngzg4_8b9d4efd-4145-49fd-a355-042b72301086/extractor/0.log" Jul 09 10:30:56.488911 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:56.488880 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589b555f76-xkg8s_c62438ba-e90e-4dc9-93f9-69115eab1e43/kube-auth-proxy/0.log" Jul 09 10:30:57.175885 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:57.175853 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5pk4g_04beab64-df60-4159-bfb8-07abe4f1c3b4/server/0.log" Jul 09 10:30:57.398631 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:57.398573 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cb598dddb-g6hv9_ed024081-9c20-476f-9e32-dd81e3cb50ea/manager/0.log" Jul 09 10:30:57.429093 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:57.429015 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5cb598dddb-g6hv9_ed024081-9c20-476f-9e32-dd81e3cb50ea/copy-manifests/0.log" Jul 09 10:30:58.239139 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:58.239104 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589b555f76-xkg8s_c62438ba-e90e-4dc9-93f9-69115eab1e43/kube-auth-proxy/0.log" Jul 09 10:30:58.957478 ip-10-0-134-220 kubenswrapper[2582]: I0709 10:30:58.957441 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c844974c9-l8kvk_cd5b19a7-4cff-43b4-bb5a-34c526b62cb1/manager/0.log"