Sep 17 19:06:35.936381 ip-10-0-143-217 systemd[1]: Starting Kubernetes Kubelet... Sep 17 19:06:36.303875 ip-10-0-143-217 kubenswrapper[2552]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 19:06:36.303875 ip-10-0-143-217 kubenswrapper[2552]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 17 19:06:36.303875 ip-10-0-143-217 kubenswrapper[2552]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 19:06:36.303875 ip-10-0-143-217 kubenswrapper[2552]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 17 19:06:36.303875 ip-10-0-143-217 kubenswrapper[2552]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 19:06:36.305975 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.305873 2552 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 17 19:06:36.309475 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309459 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 19:06:36.309475 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309475 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309479 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309482 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309485 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309488 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309492 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309494 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309497 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309502 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309511 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309515 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309518 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309520 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309523 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309526 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309528 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309531 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309533 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309536 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 19:06:36.309536 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309538 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309541 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309544 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309547 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309550 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309552 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309555 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309557 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309560 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309563 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309565 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309568 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309570 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309573 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309575 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309578 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309580 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309582 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309586 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309588 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 19:06:36.310025 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309590 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309593 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309595 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309597 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309600 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309602 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309605 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309608 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309610 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309613 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309615 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309618 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309620 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309623 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309626 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309629 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309631 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309634 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309636 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309638 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 19:06:36.310591 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309641 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309643 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309646 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309648 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309651 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309654 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309656 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309658 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309661 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309663 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309666 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309669 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309673 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309676 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309678 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309681 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309684 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309688 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309691 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 19:06:36.311094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309693 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309695 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309698 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309701 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309704 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309707 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309710 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309714 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309716 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309719 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309722 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309724 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309726 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309729 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309731 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309734 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309737 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309739 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309742 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.309744 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 19:06:36.311568 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310451 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310458 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310461 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310463 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310466 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310470 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310472 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310475 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310477 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310480 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310482 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310485 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310487 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310492 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310496 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310499 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310501 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310504 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310506 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 19:06:36.312050 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310509 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310512 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310516 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310520 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310523 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310526 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310528 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310530 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310533 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310535 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310538 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310540 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310542 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310545 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310548 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310551 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310554 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310556 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310559 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 19:06:36.312500 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310562 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310565 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310567 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310570 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310572 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310574 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310577 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310579 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310581 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310583 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310586 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310588 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310590 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310594 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310596 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310599 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310602 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310604 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310607 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310609 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 19:06:36.313002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310612 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310614 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310617 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310619 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310621 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310625 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310628 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310632 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310635 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310638 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310641 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310644 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310646 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310650 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310653 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310656 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310658 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310661 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310663 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310665 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 19:06:36.313492 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310668 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310670 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310672 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310675 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310677 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310680 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310682 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310685 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310687 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310689 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310692 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310694 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310697 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310699 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310701 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310704 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310706 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310708 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310711 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 19:06:36.314019 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310714 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.310716 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311100 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311105 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311108 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311111 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311114 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311117 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311119 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311122 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311124 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311126 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311129 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311131 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311134 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311136 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311140 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311144 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311147 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 19:06:36.314470 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311150 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311153 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311156 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311158 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311161 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311163 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311165 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311168 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311170 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311172 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311175 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311177 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311179 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311182 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311185 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311187 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311190 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311193 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311195 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 19:06:36.314936 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311199 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311203 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311206 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311208 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311211 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311213 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311216 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311218 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311221 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311223 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311225 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311228 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311230 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311232 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311235 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311237 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311239 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311242 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311244 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311246 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 19:06:36.315382 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311249 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311252 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311254 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311256 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311259 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311261 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311266 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311268 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311271 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311273 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311276 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311278 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311281 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311283 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311286 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311289 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311293 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311295 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311298 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311300 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 19:06:36.315855 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311302 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311304 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311307 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311309 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311312 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311315 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311317 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311320 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311322 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311325 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311327 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311329 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311331 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311334 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311336 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311338 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311341 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311343 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311347 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311349 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 19:06:36.316376 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311351 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311354 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.311356 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311433 2552 flags.go:64] FLAG: --address="0.0.0.0" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311480 2552 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311491 2552 flags.go:64] FLAG: --anonymous-auth="true" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311497 2552 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311503 2552 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311506 2552 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311511 2552 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311518 2552 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311522 2552 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311527 2552 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311531 2552 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311534 2552 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311537 2552 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311540 2552 flags.go:64] FLAG: --cgroup-root="" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311542 2552 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311545 2552 flags.go:64] FLAG: --client-ca-file="" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311548 2552 flags.go:64] FLAG: --cloud-provider="external" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311551 2552 flags.go:64] FLAG: --cluster-dns="[]" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311556 2552 flags.go:64] FLAG: --cluster-domain="" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311559 2552 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 17 19:06:36.316863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311561 2552 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311564 2552 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311568 2552 flags.go:64] FLAG: --container-log-max-files="5" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311571 2552 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311574 2552 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311577 2552 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311581 2552 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311584 2552 flags.go:64] FLAG: --contention-profiling="false" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311587 2552 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311590 2552 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311593 2552 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311596 2552 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311600 2552 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311602 2552 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311605 2552 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311608 2552 flags.go:64] FLAG: --enable-load-reader="false" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311611 2552 flags.go:64] FLAG: --enable-server="true" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311614 2552 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311618 2552 flags.go:64] FLAG: --event-burst="100" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311621 2552 flags.go:64] FLAG: --event-qps="50" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311624 2552 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311627 2552 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311629 2552 flags.go:64] FLAG: --eviction-hard="" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311633 2552 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 17 19:06:36.317408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311636 2552 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311638 2552 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311641 2552 flags.go:64] FLAG: --eviction-soft="" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311644 2552 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311647 2552 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311649 2552 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311652 2552 flags.go:64] FLAG: --experimental-mounter-path="" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311654 2552 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311657 2552 flags.go:64] FLAG: --fail-swap-on="true" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311660 2552 flags.go:64] FLAG: --feature-gates="" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311663 2552 flags.go:64] FLAG: --file-check-frequency="20s" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311666 2552 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311669 2552 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311672 2552 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311674 2552 flags.go:64] FLAG: --healthz-port="10248" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311677 2552 flags.go:64] FLAG: --help="false" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311680 2552 flags.go:64] FLAG: --hostname-override="ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311683 2552 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311687 2552 flags.go:64] FLAG: --http-check-frequency="20s" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311690 2552 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311693 2552 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311697 2552 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311699 2552 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 17 19:06:36.318006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311702 2552 flags.go:64] FLAG: --image-service-endpoint="" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311705 2552 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311708 2552 flags.go:64] FLAG: --kube-api-burst="100" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311711 2552 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311714 2552 flags.go:64] FLAG: --kube-api-qps="50" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311717 2552 flags.go:64] FLAG: --kube-reserved="" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311719 2552 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311722 2552 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311725 2552 flags.go:64] FLAG: --kubelet-cgroups="" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311727 2552 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311730 2552 flags.go:64] FLAG: --lock-file="" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311733 2552 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311736 2552 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311739 2552 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311744 2552 flags.go:64] FLAG: --log-json-split-stream="false" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311747 2552 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311750 2552 flags.go:64] FLAG: --log-text-split-stream="false" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311752 2552 flags.go:64] FLAG: --logging-format="text" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311755 2552 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311758 2552 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311761 2552 flags.go:64] FLAG: --manifest-url="" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311764 2552 flags.go:64] FLAG: --manifest-url-header="" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311768 2552 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311771 2552 flags.go:64] FLAG: --max-open-files="1000000" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311775 2552 flags.go:64] FLAG: --max-pods="110" Sep 17 19:06:36.318592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311778 2552 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311781 2552 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311785 2552 flags.go:64] FLAG: --memory-manager-policy="None" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311789 2552 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311791 2552 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311794 2552 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311797 2552 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311805 2552 flags.go:64] FLAG: --node-status-max-images="50" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311807 2552 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311810 2552 flags.go:64] FLAG: --oom-score-adj="-999" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311813 2552 flags.go:64] FLAG: --pod-cidr="" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311817 2552 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311822 2552 flags.go:64] FLAG: --pod-manifest-path="" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311825 2552 flags.go:64] FLAG: --pod-max-pids="-1" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311828 2552 flags.go:64] FLAG: --pods-per-core="0" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311831 2552 flags.go:64] FLAG: --port="10250" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311833 2552 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311836 2552 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04eff5a400edbb068" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311839 2552 flags.go:64] FLAG: --qos-reserved="" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311842 2552 flags.go:64] FLAG: --read-only-port="10255" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311845 2552 flags.go:64] FLAG: --register-node="true" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311848 2552 flags.go:64] FLAG: --register-with-taints="" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311851 2552 flags.go:64] FLAG: --registry-burst="10" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311854 2552 flags.go:64] FLAG: --registry-qps="5" Sep 17 19:06:36.319236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311857 2552 flags.go:64] FLAG: --reserved-cpus="" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311859 2552 flags.go:64] FLAG: --reserved-memory="" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311866 2552 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311869 2552 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311871 2552 flags.go:64] FLAG: --rotate-certificates="false" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311874 2552 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311877 2552 flags.go:64] FLAG: --runonce="false" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311880 2552 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311883 2552 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311886 2552 flags.go:64] FLAG: --seccomp-default="false" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311890 2552 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311893 2552 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311896 2552 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311913 2552 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311916 2552 flags.go:64] FLAG: --storage-driver-password="root" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311919 2552 flags.go:64] FLAG: --storage-driver-secure="false" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311922 2552 flags.go:64] FLAG: --storage-driver-table="stats" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311925 2552 flags.go:64] FLAG: --storage-driver-user="root" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311927 2552 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311930 2552 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311934 2552 flags.go:64] FLAG: --system-cgroups="" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311936 2552 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311953 2552 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311956 2552 flags.go:64] FLAG: --tls-cert-file="" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311959 2552 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 17 19:06:36.319813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311966 2552 flags.go:64] FLAG: --tls-min-version="" Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311968 2552 flags.go:64] FLAG: --tls-private-key-file="" Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311971 2552 flags.go:64] FLAG: --topology-manager-policy="none" Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311974 2552 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311976 2552 flags.go:64] FLAG: --topology-manager-scope="container" Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311979 2552 flags.go:64] FLAG: --v="2" Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311990 2552 flags.go:64] FLAG: --version="false" Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311994 2552 flags.go:64] FLAG: --vmodule="" Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.311997 2552 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.312000 2552 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312161 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312166 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312169 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312172 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312175 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312180 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312183 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312187 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312190 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312193 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312196 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312199 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 19:06:36.320427 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312201 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312204 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312207 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312209 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312212 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312214 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312225 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312227 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312230 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312233 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312235 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312237 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312240 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312242 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312245 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312247 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312250 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312252 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312255 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312257 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 19:06:36.321002 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312259 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312262 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312264 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312267 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312269 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312273 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312275 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312280 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312282 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312285 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312287 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312290 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312294 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312297 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312300 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312303 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312306 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312308 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312311 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 19:06:36.321486 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312314 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312317 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312320 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312322 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312325 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312328 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312331 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312333 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312336 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312338 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312340 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312343 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312346 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312348 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312351 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312353 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312355 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312358 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312362 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312365 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 19:06:36.322136 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312369 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312371 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312374 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312376 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312379 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312382 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312385 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312388 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312392 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312395 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312398 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312401 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312403 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312406 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312409 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312411 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312414 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312416 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312418 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 19:06:36.322974 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312421 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312423 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312426 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312428 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312430 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312433 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312435 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312438 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.312440 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.312979 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.320034 2552 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.320047 2552 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.320842 2552 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.320849 2552 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320932 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 19:06:36.323503 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320938 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320942 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320945 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320948 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320950 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320953 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320956 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320959 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320961 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320964 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320966 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320969 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320971 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320974 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320976 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320979 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320982 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320985 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320987 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320990 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 19:06:36.323923 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320992 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320995 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.320997 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321000 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321002 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321005 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321008 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321010 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321016 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321020 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321024 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321027 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321030 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321032 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321035 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321038 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321041 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321044 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321046 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321048 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 19:06:36.324398 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321051 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321053 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321056 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321058 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321061 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321063 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321066 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321069 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321071 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321074 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321076 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321079 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321081 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321084 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321086 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321089 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321091 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321094 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321096 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321098 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 19:06:36.324924 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321101 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321104 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321107 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321110 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321112 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321115 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321117 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321120 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321122 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321126 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321129 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321132 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321134 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321137 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321139 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321142 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321145 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321149 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321152 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 19:06:36.325436 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321155 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321158 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321160 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321163 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321165 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321168 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321170 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321173 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321175 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321177 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321180 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321182 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321185 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321188 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321191 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321194 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321197 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321199 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 19:06:36.325984 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321201 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.321207 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321329 2552 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321334 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321337 2552 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321340 2552 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321342 2552 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321345 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321348 2552 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321350 2552 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321353 2552 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321356 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321358 2552 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321361 2552 feature_gate.go:328] unrecognized feature gate: Example Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321363 2552 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 19:06:36.326415 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321366 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321368 2552 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321371 2552 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321373 2552 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321375 2552 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321378 2552 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321380 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321383 2552 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321385 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321388 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321390 2552 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321393 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321395 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321398 2552 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321400 2552 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321403 2552 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321406 2552 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321408 2552 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321411 2552 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 19:06:36.326771 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321414 2552 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321417 2552 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321421 2552 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321424 2552 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321427 2552 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321430 2552 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321433 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321435 2552 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321438 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321441 2552 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321444 2552 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321446 2552 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321449 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321451 2552 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321454 2552 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321456 2552 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321459 2552 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321461 2552 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321464 2552 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321466 2552 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 19:06:36.327242 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321469 2552 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321471 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321474 2552 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321478 2552 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321480 2552 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321484 2552 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321488 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321491 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321494 2552 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321498 2552 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321501 2552 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321503 2552 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321506 2552 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321508 2552 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321511 2552 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321514 2552 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321517 2552 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321519 2552 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321522 2552 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 19:06:36.327708 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321524 2552 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321527 2552 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321530 2552 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321533 2552 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321535 2552 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321539 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321542 2552 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321545 2552 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321547 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321550 2552 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321552 2552 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321555 2552 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321557 2552 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321559 2552 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321562 2552 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321564 2552 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321567 2552 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321569 2552 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321572 2552 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321574 2552 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 19:06:36.328225 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321576 2552 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321579 2552 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321581 2552 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321584 2552 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321587 2552 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321589 2552 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321592 2552 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:36.321594 2552 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.321599 2552 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.321712 2552 server.go:962] "Client rotation is on, will bootstrap in background" Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.324782 2552 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.325915 2552 server.go:1019] "Starting client certificate rotation" Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.326012 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 19:06:36.328713 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.326044 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 19:06:36.329066 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.328185 2552 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 19:06:36.330678 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.330660 2552 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 19:06:36.339896 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.339876 2552 log.go:25] "Validated CRI v1 runtime API" Sep 17 19:06:36.341341 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.341327 2552 log.go:25] "Validated CRI v1 image API" Sep 17 19:06:36.342188 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.342161 2552 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 17 19:06:36.347850 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.347828 2552 fs.go:135] Filesystem UUIDs: map[4cb432e1-331f-4276-a8e7-9b438c3d96f6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c2660a58-0cf3-4973-852e-cfc30f50cf82:/dev/nvme0n1p3] Sep 17 19:06:36.347934 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.347852 2552 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 17 19:06:36.353611 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.353504 2552 manager.go:216] Machine: {Timestamp:2026-09-17 19:06:36.352330268 +0000 UTC m=+0.305951618 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3199340 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a2139bbc7235c55d09d6288b1b94b SystemUUID:ec2a2139-bbc7-235c-55d0-9d6288b1b94b BootID:a8cd0201-889d-4d4b-a232-38c0b17cc683 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fd:36:44:49 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fd:36:44:49 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:f5:89:66:c8:6e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 17 19:06:36.353611 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.353607 2552 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 17 19:06:36.353722 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.353714 2552 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 17 19:06:36.355025 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.355010 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 19:06:36.356795 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.356760 2552 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 17 19:06:36.356944 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.356800 2552 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-217.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 17 19:06:36.357927 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.357918 2552 topology_manager.go:138] "Creating topology manager with none policy" Sep 17 19:06:36.357963 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.357929 2552 container_manager_linux.go:309] "Creating device plugin manager" Sep 17 19:06:36.357963 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.357937 2552 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 19:06:36.358573 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.358564 2552 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 19:06:36.359627 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.359617 2552 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 17 19:06:36.362012 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.362001 2552 state_mem.go:36] "Initialized new in-memory state store" Sep 17 19:06:36.362132 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.362123 2552 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 17 19:06:36.362163 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.362152 2552 kubelet.go:486] "Attempting to sync node with API server" Sep 17 19:06:36.362190 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.362167 2552 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 17 19:06:36.362190 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.362181 2552 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 17 19:06:36.362190 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.362190 2552 kubelet.go:398] "Adding apiserver pod source" Sep 17 19:06:36.362264 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.362198 2552 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 17 19:06:36.362668 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.362649 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 19:06:36.362736 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.362683 2552 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 19:06:36.366589 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.366567 2552 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 17 19:06:36.368498 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.368485 2552 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 17 19:06:36.368546 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.368514 2552 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 17 19:06:36.369955 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.369941 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 17 19:06:36.369955 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.369957 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 17 19:06:36.370066 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.369963 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 17 19:06:36.370066 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.369968 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 17 19:06:36.370066 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.369976 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 17 19:06:36.370066 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.369981 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 17 19:06:36.370066 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.369993 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 17 19:06:36.370066 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.369998 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 17 19:06:36.370066 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.370004 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 17 19:06:36.370066 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.370009 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 17 19:06:36.370066 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.370017 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 17 19:06:36.370066 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.370022 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 17 19:06:36.370705 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.370695 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 17 19:06:36.370705 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.370704 2552 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 17 19:06:36.373325 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.373297 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-217.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 17 19:06:36.373392 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.373323 2552 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-217.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 17 19:06:36.373392 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.373297 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 17 19:06:36.374362 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.374349 2552 server.go:1268] "Started kubelet" Sep 17 19:06:36.374477 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.374417 2552 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 17 19:06:36.374645 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.374439 2552 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 17 19:06:36.374645 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.374634 2552 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 17 19:06:36.375090 ip-10-0-143-217 systemd[1]: Started Kubernetes Kubelet. Sep 17 19:06:36.375647 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.375629 2552 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 17 19:06:36.377393 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.377377 2552 server.go:310] "Adding debug handlers to kubelet server" Sep 17 19:06:36.378914 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.378875 2552 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 17 19:06:36.380844 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.380828 2552 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 17 19:06:36.381317 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.381296 2552 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 17 19:06:36.381483 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.381466 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-217.ec2.internal\" not found" Sep 17 19:06:36.381891 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.381870 2552 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 17 19:06:36.381891 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.381872 2552 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 17 19:06:36.382025 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.381926 2552 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 17 19:06:36.382025 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.382015 2552 reconstruct.go:97] "Volume reconstruction finished" Sep 17 19:06:36.382025 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.382021 2552 reconciler.go:29] "Reconciler: start to sync state" Sep 17 19:06:36.382109 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.382052 2552 factory.go:55] Registering systemd factory Sep 17 19:06:36.382109 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.382066 2552 factory.go:223] Registration of the systemd container factory successfully Sep 17 19:06:36.382285 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.382272 2552 factory.go:153] Registering CRI-O factory Sep 17 19:06:36.382324 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.382286 2552 factory.go:223] Registration of the crio container factory successfully Sep 17 19:06:36.382355 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.382337 2552 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 17 19:06:36.382386 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.382354 2552 factory.go:103] Registering Raw factory Sep 17 19:06:36.382386 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.382366 2552 manager.go:1196] Started watching for new ooms in manager Sep 17 19:06:36.382722 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.382709 2552 manager.go:349] Starting recovery of all containers Sep 17 19:06:36.389071 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.389043 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 17 19:06:36.389860 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.389699 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bp476" Sep 17 19:06:36.391329 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.391296 2552 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-217.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 17 19:06:36.391451 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.391420 2552 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 17 19:06:36.392582 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.391570 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-217.ec2.internal.18d63132d85afca1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-217.ec2.internal,UID:ip-10-0-143-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-217.ec2.internal,},FirstTimestamp:2026-09-17 19:06:36.374318241 +0000 UTC m=+0.327939590,LastTimestamp:2026-09-17 19:06:36.374318241 +0000 UTC m=+0.327939590,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-217.ec2.internal,}" Sep 17 19:06:36.394619 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.394601 2552 manager.go:354] Recovery completed Sep 17 19:06:36.396393 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.396364 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bp476" Sep 17 19:06:36.400139 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.400126 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:36.402032 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.402014 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:36.402107 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.402045 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:36.402107 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.402058 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:36.402569 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.402555 2552 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 17 19:06:36.402632 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.402570 2552 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 17 19:06:36.402632 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.402586 2552 state_mem.go:36] "Initialized new in-memory state store" Sep 17 19:06:36.402632 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.402591 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:36.403394 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.403381 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:36.403440 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.403407 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:36.403440 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.403418 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:36.403629 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.403616 2552 policy_none.go:49] "None policy: Start" Sep 17 19:06:36.403686 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.403636 2552 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 17 19:06:36.403686 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.403650 2552 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 17 19:06:36.404583 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.404571 2552 policy_none.go:47] "Start" Sep 17 19:06:36.405394 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.405309 2552 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-217.ec2.internal.18d63132da01d8e1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-217.ec2.internal,UID:ip-10-0-143-217.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-217.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-217.ec2.internal,},FirstTimestamp:2026-09-17 19:06:36.402030817 +0000 UTC m=+0.355652166,LastTimestamp:2026-09-17 19:06:36.402030817 +0000 UTC m=+0.355652166,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-217.ec2.internal,}" Sep 17 19:06:36.451530 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.451512 2552 manager.go:337] "Starting Device Plugin manager" Sep 17 19:06:36.457176 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.451555 2552 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 17 19:06:36.457176 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.451567 2552 server.go:87] "Starting device plugin registration server" Sep 17 19:06:36.457176 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.451862 2552 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 17 19:06:36.457176 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.451878 2552 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 17 19:06:36.457176 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.452028 2552 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 17 19:06:36.457176 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.452128 2552 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 17 19:06:36.457176 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.452137 2552 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 17 19:06:36.457176 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.452637 2552 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 17 19:06:36.457176 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.452681 2552 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-217.ec2.internal\" not found" Sep 17 19:06:36.509745 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.509717 2552 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 17 19:06:36.509875 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.509769 2552 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 17 19:06:36.509875 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.509789 2552 kubelet.go:2448] "Starting kubelet main sync loop" Sep 17 19:06:36.509875 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.509824 2552 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 17 19:06:36.512248 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.512231 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 19:06:36.552550 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.552523 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:36.553507 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.553491 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:36.553659 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.553526 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:36.553659 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.553543 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:36.553659 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.553581 2552 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.563123 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.563052 2552 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.563123 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.563081 2552 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-217.ec2.internal\": node \"ip-10-0-143-217.ec2.internal\" not found" Sep 17 19:06:36.590500 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.590480 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-217.ec2.internal\" not found" Sep 17 19:06:36.609934 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.609897 2552 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-217.ec2.internal"] Sep 17 19:06:36.610009 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.609978 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:36.610802 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.610787 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:36.610884 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.610816 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:36.610884 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.610831 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:36.612049 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.612036 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:36.612153 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.612134 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.612197 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.612159 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:36.612727 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.612710 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:36.612812 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.612756 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:36.612812 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.612770 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:36.612812 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.612785 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:36.612812 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.612807 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:36.612980 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.612827 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:36.613784 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.613769 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.613874 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.613794 2552 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 19:06:36.614421 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.614406 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientMemory" Sep 17 19:06:36.614497 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.614426 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 19:06:36.614497 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.614437 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeHasSufficientPID" Sep 17 19:06:36.632448 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.632428 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-217.ec2.internal\" not found" node="ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.635802 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.635777 2552 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-217.ec2.internal\" not found" node="ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.683454 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.683430 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/340d33235ef55881b15f22dfff959742-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal\" (UID: \"340d33235ef55881b15f22dfff959742\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.683531 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.683459 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/340d33235ef55881b15f22dfff959742-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal\" (UID: \"340d33235ef55881b15f22dfff959742\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.683531 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.683478 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f44ab85f08a059c30913c6fa3110b60-config\") pod \"kube-apiserver-proxy-ip-10-0-143-217.ec2.internal\" (UID: \"9f44ab85f08a059c30913c6fa3110b60\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.695201 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.695178 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-217.ec2.internal\" not found" Sep 17 19:06:36.783868 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.783845 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/340d33235ef55881b15f22dfff959742-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal\" (UID: \"340d33235ef55881b15f22dfff959742\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.783967 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.783879 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/340d33235ef55881b15f22dfff959742-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal\" (UID: \"340d33235ef55881b15f22dfff959742\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.783967 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.783896 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f44ab85f08a059c30913c6fa3110b60-config\") pod \"kube-apiserver-proxy-ip-10-0-143-217.ec2.internal\" (UID: \"9f44ab85f08a059c30913c6fa3110b60\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.784029 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.783953 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/340d33235ef55881b15f22dfff959742-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal\" (UID: \"340d33235ef55881b15f22dfff959742\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.784029 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.783953 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/340d33235ef55881b15f22dfff959742-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal\" (UID: \"340d33235ef55881b15f22dfff959742\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.784029 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.783949 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9f44ab85f08a059c30913c6fa3110b60-config\") pod \"kube-apiserver-proxy-ip-10-0-143-217.ec2.internal\" (UID: \"9f44ab85f08a059c30913c6fa3110b60\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.795948 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.795925 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-217.ec2.internal\" not found" Sep 17 19:06:36.896803 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.896706 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-217.ec2.internal\" not found" Sep 17 19:06:36.935869 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.935844 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.938757 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:36.938731 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-217.ec2.internal" Sep 17 19:06:36.997395 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:36.997367 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-217.ec2.internal\" not found" Sep 17 19:06:37.097909 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:37.097878 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-217.ec2.internal\" not found" Sep 17 19:06:37.198411 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:37.198346 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-217.ec2.internal\" not found" Sep 17 19:06:37.298894 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:37.298867 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-217.ec2.internal\" not found" Sep 17 19:06:37.326048 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.326022 2552 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 17 19:06:37.326442 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.326146 2552 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 19:06:37.399750 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:37.399716 2552 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-217.ec2.internal\" not found" Sep 17 19:06:37.399750 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.399723 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 19:01:37 +0000 UTC" deadline="2028-06-06 01:30:28.097929368 +0000 UTC" Sep 17 19:06:37.399750 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.399755 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15054h23m50.698179292s" Sep 17 19:06:37.415178 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.415154 2552 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 19:06:37.461297 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:37.461141 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod340d33235ef55881b15f22dfff959742.slice/crio-8a77164933f8fcb34a5ae9d6c9eb551acea96e5b030f182c4b4ee2dee58e6f72 WatchSource:0}: Error finding container 8a77164933f8fcb34a5ae9d6c9eb551acea96e5b030f182c4b4ee2dee58e6f72: Status 404 returned error can't find the container with id 8a77164933f8fcb34a5ae9d6c9eb551acea96e5b030f182c4b4ee2dee58e6f72 Sep 17 19:06:37.461503 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:37.461484 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f44ab85f08a059c30913c6fa3110b60.slice/crio-5f99c9a33db67362cc70216fbc39130c592d8c78eaca5105005b082e65aa08c0 WatchSource:0}: Error finding container 5f99c9a33db67362cc70216fbc39130c592d8c78eaca5105005b082e65aa08c0: Status 404 returned error can't find the container with id 5f99c9a33db67362cc70216fbc39130c592d8c78eaca5105005b082e65aa08c0 Sep 17 19:06:37.465455 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.465440 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 19:06:37.482167 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.482150 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-217.ec2.internal" Sep 17 19:06:37.485920 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.485892 2552 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 19:06:37.493422 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.493406 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 19:06:37.494341 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.494328 2552 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" Sep 17 19:06:37.504624 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.504606 2552 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 19:06:37.512634 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.512595 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" event={"ID":"340d33235ef55881b15f22dfff959742","Type":"ContainerStarted","Data":"8a77164933f8fcb34a5ae9d6c9eb551acea96e5b030f182c4b4ee2dee58e6f72"} Sep 17 19:06:37.513573 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.513556 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-217.ec2.internal" event={"ID":"9f44ab85f08a059c30913c6fa3110b60","Type":"ContainerStarted","Data":"5f99c9a33db67362cc70216fbc39130c592d8c78eaca5105005b082e65aa08c0"} Sep 17 19:06:37.719977 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:37.719884 2552 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 19:06:38.160001 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.159758 2552 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 19:06:38.363799 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.363753 2552 apiserver.go:52] "Watching apiserver" Sep 17 19:06:38.374980 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.374945 2552 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 17 19:06:38.376556 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.376524 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-5w586","kube-system/kube-apiserver-proxy-ip-10-0-143-217.ec2.internal","openshift-cluster-node-tuning-operator/tuned-psqk9","openshift-dns/node-resolver-ndhbz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal","openshift-multus/multus-additional-cni-plugins-8zwvj","openshift-network-diagnostics/network-check-target-t7jj9","openshift-ovn-kubernetes/ovnkube-node-jt5kn","kube-system/global-pull-secret-syncer-ls7s4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828","openshift-image-registry/node-ca-lsmrz","openshift-multus/multus-4nvkk","openshift-multus/network-metrics-daemon-422mp","openshift-network-operator/iptables-alerter-bnf7d"] Sep 17 19:06:38.378288 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.378268 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:38.378382 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:38.378351 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:06:38.379339 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.379314 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lsmrz" Sep 17 19:06:38.380802 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.380665 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.382495 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.382113 2552 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 17 19:06:38.382864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.382840 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ndhbz" Sep 17 19:06:38.384327 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.384308 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bnf7d" Sep 17 19:06:38.384437 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.384424 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 17 19:06:38.384663 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.384644 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pb2tw\"" Sep 17 19:06:38.385202 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.385184 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 17 19:06:38.385407 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.385390 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 17 19:06:38.385778 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.385761 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 17 19:06:38.385878 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.385796 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.386755 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.386739 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 17 19:06:38.387454 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.387211 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2mj9k\"" Sep 17 19:06:38.387521 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.387476 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 17 19:06:38.388535 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.387673 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 17 19:06:38.388535 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.387994 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 17 19:06:38.388535 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.388018 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 17 19:06:38.388535 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.388307 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r469w\"" Sep 17 19:06:38.388535 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.388316 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7ldvk\"" Sep 17 19:06:38.388535 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.388367 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 17 19:06:38.388535 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.388386 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:38.388535 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:38.388470 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:06:38.388878 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.388799 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:38.388878 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:38.388860 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:06:38.390259 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.390239 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.391758 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.391737 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.393248 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.393230 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5w586" Sep 17 19:06:38.393854 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.393827 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-sysconfig\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394021 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.393858 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-host\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394021 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.393879 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx8vv\" (UniqueName: \"kubernetes.io/projected/27709855-de56-4854-a822-e6044d98d1a4-kube-api-access-tx8vv\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394021 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.393904 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac559d16-e42b-4873-bd8e-abda5c4137fd-tmp-dir\") pod \"node-resolver-ndhbz\" (UID: \"ac559d16-e42b-4873-bd8e-abda5c4137fd\") " pod="openshift-dns/node-resolver-ndhbz" Sep 17 19:06:38.394021 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.393930 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a-host\") pod \"node-ca-lsmrz\" (UID: \"4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a\") " pod="openshift-image-registry/node-ca-lsmrz" Sep 17 19:06:38.394021 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.393957 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-modprobe-d\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394021 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394005 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-kubernetes\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394357 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394048 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-sysctl-d\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394357 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394074 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-sysctl-conf\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394357 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394120 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-sys\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394357 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394150 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-lib-modules\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394357 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394178 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a-serviceca\") pod \"node-ca-lsmrz\" (UID: \"4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a\") " pod="openshift-image-registry/node-ca-lsmrz" Sep 17 19:06:38.394357 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394203 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvkbt\" (UniqueName: \"kubernetes.io/projected/4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a-kube-api-access-lvkbt\") pod \"node-ca-lsmrz\" (UID: \"4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a\") " pod="openshift-image-registry/node-ca-lsmrz" Sep 17 19:06:38.394357 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394241 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-run\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394357 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394295 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-var-lib-kubelet\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394357 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394321 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/27709855-de56-4854-a822-e6044d98d1a4-etc-tuned\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394357 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394347 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27709855-de56-4854-a822-e6044d98d1a4-tmp\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394778 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394374 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac559d16-e42b-4873-bd8e-abda5c4137fd-hosts-file\") pod \"node-resolver-ndhbz\" (UID: \"ac559d16-e42b-4873-bd8e-abda5c4137fd\") " pod="openshift-dns/node-resolver-ndhbz" Sep 17 19:06:38.394778 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394409 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-systemd\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.394778 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394433 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qszmj\" (UniqueName: \"kubernetes.io/projected/ac559d16-e42b-4873-bd8e-abda5c4137fd-kube-api-access-qszmj\") pod \"node-resolver-ndhbz\" (UID: \"ac559d16-e42b-4873-bd8e-abda5c4137fd\") " pod="openshift-dns/node-resolver-ndhbz" Sep 17 19:06:38.394778 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394462 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v48sq\" (UniqueName: \"kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq\") pod \"network-check-target-t7jj9\" (UID: \"45c9424d-9eba-4232-9e83-85dff9e0132f\") " pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:38.394991 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.394819 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.397491 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.397468 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 17 19:06:38.397624 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.397587 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 17 19:06:38.397800 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.397769 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 17 19:06:38.397800 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.397784 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 17 19:06:38.397956 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.397776 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 17 19:06:38.398037 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.397981 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 17 19:06:38.398037 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.398001 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 17 19:06:38.398148 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.398053 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-644x6\"" Sep 17 19:06:38.398148 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.398101 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 17 19:06:38.399005 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.398972 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 17 19:06:38.399005 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.398972 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 17 19:06:38.399197 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.399091 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 17 19:06:38.399197 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.399119 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 17 19:06:38.399197 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.399160 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 17 19:06:38.399332 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.399212 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 17 19:06:38.399332 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.399232 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xg272\"" Sep 17 19:06:38.399426 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.399353 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 17 19:06:38.399476 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.399439 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-snxf2\"" Sep 17 19:06:38.399476 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.399462 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-87hmb\"" Sep 17 19:06:38.399579 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.399490 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-9xlxm\"" Sep 17 19:06:38.399579 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.399549 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 17 19:06:38.399811 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.399782 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 17 19:06:38.406723 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.406706 2552 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 19:06:38.445377 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.445354 2552 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-k5gh8" Sep 17 19:06:38.457813 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.457790 2552 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-k5gh8" Sep 17 19:06:38.482916 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.482886 2552 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 17 19:06:38.494768 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.494733 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c439c32d-c42d-4795-8e5e-d4dcf5479e32-ovn-node-metrics-cert\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.494960 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.494779 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-run-openvswitch\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.494960 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.494808 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-node-log\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.494960 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.494833 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-cni-bin\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.494960 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.494857 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d78749d7-d120-4da2-a5fd-29d6bbe672c3-konnectivity-ca\") pod \"konnectivity-agent-5w586\" (UID: \"d78749d7-d120-4da2-a5fd-29d6bbe672c3\") " pod="kube-system/konnectivity-agent-5w586" Sep 17 19:06:38.494960 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.494891 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-sys\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.495212 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.494963 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxq4r\" (UniqueName: \"kubernetes.io/projected/c439c32d-c42d-4795-8e5e-d4dcf5479e32-kube-api-access-xxq4r\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.495212 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.494990 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-sys\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.495212 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495002 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a-serviceca\") pod \"node-ca-lsmrz\" (UID: \"4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a\") " pod="openshift-image-registry/node-ca-lsmrz" Sep 17 19:06:38.495212 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495032 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvkbt\" (UniqueName: \"kubernetes.io/projected/4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a-kube-api-access-lvkbt\") pod \"node-ca-lsmrz\" (UID: \"4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a\") " pod="openshift-image-registry/node-ca-lsmrz" Sep 17 19:06:38.495212 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495059 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27709855-de56-4854-a822-e6044d98d1a4-tmp\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.495212 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495189 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/441acc97-89e4-47a7-9ea1-923c8e7e99a7-iptables-alerter-script\") pod \"iptables-alerter-bnf7d\" (UID: \"441acc97-89e4-47a7-9ea1-923c8e7e99a7\") " pod="openshift-network-operator/iptables-alerter-bnf7d" Sep 17 19:06:38.495504 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495234 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e96469d5-497d-4b64-ad08-465aa483e44a-cnibin\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.495504 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495261 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-var-lib-openvswitch\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.495504 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495287 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-run-netns\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.495504 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495308 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c439c32d-c42d-4795-8e5e-d4dcf5479e32-env-overrides\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.495504 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495337 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-sysconfig\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.495504 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495362 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx8vv\" (UniqueName: \"kubernetes.io/projected/27709855-de56-4854-a822-e6044d98d1a4-kube-api-access-tx8vv\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.495504 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495385 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-multus-cni-dir\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.495504 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495412 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-run-k8s-cni-cncf-io\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.495504 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495437 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6zhs\" (UniqueName: \"kubernetes.io/projected/441acc97-89e4-47a7-9ea1-923c8e7e99a7-kube-api-access-d6zhs\") pod \"iptables-alerter-bnf7d\" (UID: \"441acc97-89e4-47a7-9ea1-923c8e7e99a7\") " pod="openshift-network-operator/iptables-alerter-bnf7d" Sep 17 19:06:38.495504 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495458 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-slash\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.495504 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495479 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-run-ovn-kubernetes\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.495504 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495500 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c439c32d-c42d-4795-8e5e-d4dcf5479e32-ovnkube-config\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495520 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-sysctl-conf\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495529 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a-serviceca\") pod \"node-ca-lsmrz\" (UID: \"4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a\") " pod="openshift-image-registry/node-ca-lsmrz" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495560 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-sys-fs\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495751 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-sysctl-conf\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495814 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69fv9\" (UniqueName: \"kubernetes.io/projected/6de92e0b-f310-4697-81fc-7293d809946d-kube-api-access-69fv9\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495837 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-sysconfig\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495884 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-multus-socket-dir-parent\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495916 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c439c32d-c42d-4795-8e5e-d4dcf5479e32-ovnkube-script-lib\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495922 2552 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.495955 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-device-dir\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496044 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56c11fe2-25e7-4bb2-b789-ea4178f73d25-cni-binary-copy\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496069 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:38.496095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496097 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-systemd\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496122 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-registration-dir\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496150 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e96469d5-497d-4b64-ad08-465aa483e44a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496167 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-systemd\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496172 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbck2\" (UniqueName: \"kubernetes.io/projected/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-kube-api-access-nbck2\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496224 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-host\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496252 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-var-lib-cni-multus\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496300 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-host\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496302 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56c11fe2-25e7-4bb2-b789-ea4178f73d25-multus-daemon-config\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496327 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-etc-kubernetes\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496364 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-run-systemd\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496415 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-cni-netd\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496449 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a-host\") pod \"node-ca-lsmrz\" (UID: \"4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a\") " pod="openshift-image-registry/node-ca-lsmrz" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496468 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-modprobe-d\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496484 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-var-lib-kubelet\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496504 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-etc-selinux\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496495 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a-host\") pod \"node-ca-lsmrz\" (UID: \"4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a\") " pod="openshift-image-registry/node-ca-lsmrz" Sep 17 19:06:38.496660 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496560 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-etc-openvswitch\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496584 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-var-lib-kubelet\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496584 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-run\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496613 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-modprobe-d\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496623 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-var-lib-cni-bin\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496646 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbr8w\" (UniqueName: \"kubernetes.io/projected/e96469d5-497d-4b64-ad08-465aa483e44a-kube-api-access-dbr8w\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496663 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-systemd-units\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496670 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-run\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496677 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d78749d7-d120-4da2-a5fd-29d6bbe672c3-agent-certs\") pod \"konnectivity-agent-5w586\" (UID: \"d78749d7-d120-4da2-a5fd-29d6bbe672c3\") " pod="kube-system/konnectivity-agent-5w586" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496696 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v48sq\" (UniqueName: \"kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq\") pod \"network-check-target-t7jj9\" (UID: \"45c9424d-9eba-4232-9e83-85dff9e0132f\") " pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496721 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496745 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-socket-dir\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496760 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-system-cni-dir\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496785 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e96469d5-497d-4b64-ad08-465aa483e44a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496824 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-run-ovn\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496857 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac559d16-e42b-4873-bd8e-abda5c4137fd-tmp-dir\") pod \"node-resolver-ndhbz\" (UID: \"ac559d16-e42b-4873-bd8e-abda5c4137fd\") " pod="openshift-dns/node-resolver-ndhbz" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.496924 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:38.497290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497002 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-log-socket\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497040 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497069 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c5ac3838-c267-4741-8925-8c2e90745194-dbus\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497097 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-kubernetes\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497124 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-sysctl-d\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497150 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-lib-modules\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497164 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac559d16-e42b-4873-bd8e-abda5c4137fd-tmp-dir\") pod \"node-resolver-ndhbz\" (UID: \"ac559d16-e42b-4873-bd8e-abda5c4137fd\") " pod="openshift-dns/node-resolver-ndhbz" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497174 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac559d16-e42b-4873-bd8e-abda5c4137fd-hosts-file\") pod \"node-resolver-ndhbz\" (UID: \"ac559d16-e42b-4873-bd8e-abda5c4137fd\") " pod="openshift-dns/node-resolver-ndhbz" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497217 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-kubernetes\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497218 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsxq8\" (UniqueName: \"kubernetes.io/projected/56c11fe2-25e7-4bb2-b789-ea4178f73d25-kube-api-access-lsxq8\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497262 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/441acc97-89e4-47a7-9ea1-923c8e7e99a7-host-slash\") pod \"iptables-alerter-bnf7d\" (UID: \"441acc97-89e4-47a7-9ea1-923c8e7e99a7\") " pod="openshift-network-operator/iptables-alerter-bnf7d" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497286 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e96469d5-497d-4b64-ad08-465aa483e44a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497310 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/27709855-de56-4854-a822-e6044d98d1a4-etc-tuned\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497334 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c5ac3838-c267-4741-8925-8c2e90745194-kubelet-config\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497341 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-etc-sysctl-d\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497338 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/27709855-de56-4854-a822-e6044d98d1a4-lib-modules\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497266 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac559d16-e42b-4873-bd8e-abda5c4137fd-hosts-file\") pod \"node-resolver-ndhbz\" (UID: \"ac559d16-e42b-4873-bd8e-abda5c4137fd\") " pod="openshift-dns/node-resolver-ndhbz" Sep 17 19:06:38.497864 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497359 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-hostroot\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.498571 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497384 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-run-multus-certs\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.498571 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497401 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e96469d5-497d-4b64-ad08-465aa483e44a-cni-binary-copy\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.498571 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497424 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-kubelet\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.498571 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497440 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-run-netns\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.498571 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497456 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qszmj\" (UniqueName: \"kubernetes.io/projected/ac559d16-e42b-4873-bd8e-abda5c4137fd-kube-api-access-qszmj\") pod \"node-resolver-ndhbz\" (UID: \"ac559d16-e42b-4873-bd8e-abda5c4137fd\") " pod="openshift-dns/node-resolver-ndhbz" Sep 17 19:06:38.498571 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497471 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-cnibin\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.498571 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497499 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-var-lib-kubelet\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.498571 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497527 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-multus-conf-dir\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.498571 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.497567 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e96469d5-497d-4b64-ad08-465aa483e44a-system-cni-dir\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.500426 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.500409 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27709855-de56-4854-a822-e6044d98d1a4-tmp\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.501142 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.501124 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/27709855-de56-4854-a822-e6044d98d1a4-etc-tuned\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.502508 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:38.502485 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:06:38.502640 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:38.502513 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:06:38.502640 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:38.502526 2552 projected.go:196] Error preparing data for projected volume kube-api-access-v48sq for pod openshift-network-diagnostics/network-check-target-t7jj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:38.502640 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:38.502625 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq podName:45c9424d-9eba-4232-9e83-85dff9e0132f nodeName:}" failed. No retries permitted until 2026-09-17 19:06:39.002577231 +0000 UTC m=+2.956198574 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-v48sq" (UniqueName: "kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq") pod "network-check-target-t7jj9" (UID: "45c9424d-9eba-4232-9e83-85dff9e0132f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:38.504921 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.504879 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvkbt\" (UniqueName: \"kubernetes.io/projected/4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a-kube-api-access-lvkbt\") pod \"node-ca-lsmrz\" (UID: \"4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a\") " pod="openshift-image-registry/node-ca-lsmrz" Sep 17 19:06:38.505327 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.505304 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx8vv\" (UniqueName: \"kubernetes.io/projected/27709855-de56-4854-a822-e6044d98d1a4-kube-api-access-tx8vv\") pod \"tuned-psqk9\" (UID: \"27709855-de56-4854-a822-e6044d98d1a4\") " pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.506159 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.506138 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qszmj\" (UniqueName: \"kubernetes.io/projected/ac559d16-e42b-4873-bd8e-abda5c4137fd-kube-api-access-qszmj\") pod \"node-resolver-ndhbz\" (UID: \"ac559d16-e42b-4873-bd8e-abda5c4137fd\") " pod="openshift-dns/node-resolver-ndhbz" Sep 17 19:06:38.598678 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598638 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c5ac3838-c267-4741-8925-8c2e90745194-kubelet-config\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:38.598678 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598683 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-hostroot\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598706 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-run-multus-certs\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598768 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-run-multus-certs\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598780 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-hostroot\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598790 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c5ac3838-c267-4741-8925-8c2e90745194-kubelet-config\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598801 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e96469d5-497d-4b64-ad08-465aa483e44a-cni-binary-copy\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598848 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-kubelet\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598872 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-run-netns\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598898 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-cnibin\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598927 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-kubelet\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598948 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-var-lib-kubelet\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598975 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-multus-conf-dir\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598977 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-cnibin\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.598982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598986 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-var-lib-kubelet\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598934 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-run-netns\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.598998 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e96469d5-497d-4b64-ad08-465aa483e44a-system-cni-dir\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599024 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c439c32d-c42d-4795-8e5e-d4dcf5479e32-ovn-node-metrics-cert\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599049 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-run-openvswitch\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599030 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-multus-conf-dir\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599077 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-node-log\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599117 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-run-openvswitch\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599104 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-cni-bin\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599166 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d78749d7-d120-4da2-a5fd-29d6bbe672c3-konnectivity-ca\") pod \"konnectivity-agent-5w586\" (UID: \"d78749d7-d120-4da2-a5fd-29d6bbe672c3\") " pod="kube-system/konnectivity-agent-5w586" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599195 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxq4r\" (UniqueName: \"kubernetes.io/projected/c439c32d-c42d-4795-8e5e-d4dcf5479e32-kube-api-access-xxq4r\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599193 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-node-log\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599207 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e96469d5-497d-4b64-ad08-465aa483e44a-system-cni-dir\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599196 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-cni-bin\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599226 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/441acc97-89e4-47a7-9ea1-923c8e7e99a7-iptables-alerter-script\") pod \"iptables-alerter-bnf7d\" (UID: \"441acc97-89e4-47a7-9ea1-923c8e7e99a7\") " pod="openshift-network-operator/iptables-alerter-bnf7d" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599258 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e96469d5-497d-4b64-ad08-465aa483e44a-cnibin\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599282 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-var-lib-openvswitch\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.599653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599308 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-run-netns\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599333 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c439c32d-c42d-4795-8e5e-d4dcf5479e32-env-overrides\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599342 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e96469d5-497d-4b64-ad08-465aa483e44a-cnibin\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599359 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-multus-cni-dir\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599384 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-run-k8s-cni-cncf-io\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599389 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-run-netns\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599410 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6zhs\" (UniqueName: \"kubernetes.io/projected/441acc97-89e4-47a7-9ea1-923c8e7e99a7-kube-api-access-d6zhs\") pod \"iptables-alerter-bnf7d\" (UID: \"441acc97-89e4-47a7-9ea1-923c8e7e99a7\") " pod="openshift-network-operator/iptables-alerter-bnf7d" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599428 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e96469d5-497d-4b64-ad08-465aa483e44a-cni-binary-copy\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599431 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-var-lib-openvswitch\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599447 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-slash\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599482 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-run-ovn-kubernetes\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599493 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-slash\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599503 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c439c32d-c42d-4795-8e5e-d4dcf5479e32-ovnkube-config\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599531 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-run-ovn-kubernetes\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599554 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-sys-fs\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599573 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-multus-cni-dir\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599582 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69fv9\" (UniqueName: \"kubernetes.io/projected/6de92e0b-f310-4697-81fc-7293d809946d-kube-api-access-69fv9\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.600499 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599609 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-multus-socket-dir-parent\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599624 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-run-k8s-cni-cncf-io\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599634 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c439c32d-c42d-4795-8e5e-d4dcf5479e32-ovnkube-script-lib\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599658 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-device-dir\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599684 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56c11fe2-25e7-4bb2-b789-ea4178f73d25-cni-binary-copy\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599709 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599735 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-registration-dir\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599761 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e96469d5-497d-4b64-ad08-465aa483e44a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599788 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbck2\" (UniqueName: \"kubernetes.io/projected/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-kube-api-access-nbck2\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599816 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-var-lib-cni-multus\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599831 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/441acc97-89e4-47a7-9ea1-923c8e7e99a7-iptables-alerter-script\") pod \"iptables-alerter-bnf7d\" (UID: \"441acc97-89e4-47a7-9ea1-923c8e7e99a7\") " pod="openshift-network-operator/iptables-alerter-bnf7d" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599849 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56c11fe2-25e7-4bb2-b789-ea4178f73d25-multus-daemon-config\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599875 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-etc-kubernetes\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599884 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-sys-fs\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599910 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-run-systemd\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599935 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-cni-netd\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599854 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d78749d7-d120-4da2-a5fd-29d6bbe672c3-konnectivity-ca\") pod \"konnectivity-agent-5w586\" (UID: \"d78749d7-d120-4da2-a5fd-29d6bbe672c3\") " pod="kube-system/konnectivity-agent-5w586" Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:38.599993 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:38.601280 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600012 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-multus-socket-dir-parent\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599971 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-etc-selinux\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600043 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c439c32d-c42d-4795-8e5e-d4dcf5479e32-ovnkube-config\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600060 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-etc-openvswitch\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:38.600070 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs podName:366e0b69-56de-45e9-9a8b-87f0a4c8b4bf nodeName:}" failed. No retries permitted until 2026-09-17 19:06:39.100052758 +0000 UTC m=+3.053674141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs") pod "network-metrics-daemon-422mp" (UID: "366e0b69-56de-45e9-9a8b-87f0a4c8b4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599946 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-registration-dir\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600093 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-etc-openvswitch\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600105 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-etc-selinux\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600149 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-var-lib-cni-bin\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600178 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbr8w\" (UniqueName: \"kubernetes.io/projected/e96469d5-497d-4b64-ad08-465aa483e44a-kube-api-access-dbr8w\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600205 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-systemd-units\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600217 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-var-lib-cni-multus\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600237 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d78749d7-d120-4da2-a5fd-29d6bbe672c3-agent-certs\") pod \"konnectivity-agent-5w586\" (UID: \"d78749d7-d120-4da2-a5fd-29d6bbe672c3\") " pod="kube-system/konnectivity-agent-5w586" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600286 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-cni-netd\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.599875 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c439c32d-c42d-4795-8e5e-d4dcf5479e32-env-overrides\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600375 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/56c11fe2-25e7-4bb2-b789-ea4178f73d25-cni-binary-copy\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600377 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-run-systemd\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.602063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600432 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-host-var-lib-cni-bin\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600434 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-device-dir\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600482 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-etc-kubernetes\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600524 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-systemd-units\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600547 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c439c32d-c42d-4795-8e5e-d4dcf5479e32-ovnkube-script-lib\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600579 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600606 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-socket-dir\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600632 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-system-cni-dir\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600657 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e96469d5-497d-4b64-ad08-465aa483e44a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600683 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-run-ovn\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600692 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/56c11fe2-25e7-4bb2-b789-ea4178f73d25-multus-daemon-config\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600700 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/56c11fe2-25e7-4bb2-b789-ea4178f73d25-system-cni-dir\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600710 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600752 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600760 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e96469d5-497d-4b64-ad08-465aa483e44a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600762 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6de92e0b-f310-4697-81fc-7293d809946d-socket-dir\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600787 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-log-socket\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.603044 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600801 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-run-ovn\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600815 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600840 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-log-socket\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:38.600807 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600875 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c5ac3838-c267-4741-8925-8c2e90745194-dbus\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600874 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c439c32d-c42d-4795-8e5e-d4dcf5479e32-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:38.600932 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret podName:c5ac3838-c267-4741-8925-8c2e90745194 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:39.100897115 +0000 UTC m=+3.054518456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret") pod "global-pull-secret-syncer-ls7s4" (UID: "c5ac3838-c267-4741-8925-8c2e90745194") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600951 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsxq8\" (UniqueName: \"kubernetes.io/projected/56c11fe2-25e7-4bb2-b789-ea4178f73d25-kube-api-access-lsxq8\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.600952 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c5ac3838-c267-4741-8925-8c2e90745194-dbus\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.601004 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/441acc97-89e4-47a7-9ea1-923c8e7e99a7-host-slash\") pod \"iptables-alerter-bnf7d\" (UID: \"441acc97-89e4-47a7-9ea1-923c8e7e99a7\") " pod="openshift-network-operator/iptables-alerter-bnf7d" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.601042 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/441acc97-89e4-47a7-9ea1-923c8e7e99a7-host-slash\") pod \"iptables-alerter-bnf7d\" (UID: \"441acc97-89e4-47a7-9ea1-923c8e7e99a7\") " pod="openshift-network-operator/iptables-alerter-bnf7d" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.601045 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e96469d5-497d-4b64-ad08-465aa483e44a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.601128 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e96469d5-497d-4b64-ad08-465aa483e44a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.601215 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e96469d5-497d-4b64-ad08-465aa483e44a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.601878 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c439c32d-c42d-4795-8e5e-d4dcf5479e32-ovn-node-metrics-cert\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.603901 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.603208 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d78749d7-d120-4da2-a5fd-29d6bbe672c3-agent-certs\") pod \"konnectivity-agent-5w586\" (UID: \"d78749d7-d120-4da2-a5fd-29d6bbe672c3\") " pod="kube-system/konnectivity-agent-5w586" Sep 17 19:06:38.610829 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.610800 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxq4r\" (UniqueName: \"kubernetes.io/projected/c439c32d-c42d-4795-8e5e-d4dcf5479e32-kube-api-access-xxq4r\") pod \"ovnkube-node-jt5kn\" (UID: \"c439c32d-c42d-4795-8e5e-d4dcf5479e32\") " pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.611402 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.611375 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69fv9\" (UniqueName: \"kubernetes.io/projected/6de92e0b-f310-4697-81fc-7293d809946d-kube-api-access-69fv9\") pod \"aws-ebs-csi-driver-node-df828\" (UID: \"6de92e0b-f310-4697-81fc-7293d809946d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.611503 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.611381 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbr8w\" (UniqueName: \"kubernetes.io/projected/e96469d5-497d-4b64-ad08-465aa483e44a-kube-api-access-dbr8w\") pod \"multus-additional-cni-plugins-8zwvj\" (UID: \"e96469d5-497d-4b64-ad08-465aa483e44a\") " pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.612306 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.612262 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbck2\" (UniqueName: \"kubernetes.io/projected/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-kube-api-access-nbck2\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:38.612898 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.612816 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6zhs\" (UniqueName: \"kubernetes.io/projected/441acc97-89e4-47a7-9ea1-923c8e7e99a7-kube-api-access-d6zhs\") pod \"iptables-alerter-bnf7d\" (UID: \"441acc97-89e4-47a7-9ea1-923c8e7e99a7\") " pod="openshift-network-operator/iptables-alerter-bnf7d" Sep 17 19:06:38.613994 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.613974 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsxq8\" (UniqueName: \"kubernetes.io/projected/56c11fe2-25e7-4bb2-b789-ea4178f73d25-kube-api-access-lsxq8\") pod \"multus-4nvkk\" (UID: \"56c11fe2-25e7-4bb2-b789-ea4178f73d25\") " pod="openshift-multus/multus-4nvkk" Sep 17 19:06:38.691657 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.691573 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lsmrz" Sep 17 19:06:38.699866 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.699849 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-psqk9" Sep 17 19:06:38.708021 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.707999 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ndhbz" Sep 17 19:06:38.716883 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.716859 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bnf7d" Sep 17 19:06:38.722959 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.722937 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8zwvj" Sep 17 19:06:38.729795 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.729775 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:06:38.736700 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.736680 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" Sep 17 19:06:38.743391 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.743373 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5w586" Sep 17 19:06:38.747986 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:38.747966 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4nvkk" Sep 17 19:06:39.003989 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.003897 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v48sq\" (UniqueName: \"kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq\") pod \"network-check-target-t7jj9\" (UID: \"45c9424d-9eba-4232-9e83-85dff9e0132f\") " pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:39.004135 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:39.004042 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:06:39.004135 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:39.004067 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:06:39.004135 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:39.004077 2552 projected.go:196] Error preparing data for projected volume kube-api-access-v48sq for pod openshift-network-diagnostics/network-check-target-t7jj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:39.004255 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:39.004139 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq podName:45c9424d-9eba-4232-9e83-85dff9e0132f nodeName:}" failed. No retries permitted until 2026-09-17 19:06:40.004125049 +0000 UTC m=+3.957746387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-v48sq" (UniqueName: "kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq") pod "network-check-target-t7jj9" (UID: "45c9424d-9eba-4232-9e83-85dff9e0132f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:39.105215 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.105180 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:39.105372 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.105245 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:39.105372 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:39.105345 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:39.105372 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:39.105350 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:39.105513 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:39.105415 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs podName:366e0b69-56de-45e9-9a8b-87f0a4c8b4bf nodeName:}" failed. No retries permitted until 2026-09-17 19:06:40.105395476 +0000 UTC m=+4.059016839 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs") pod "network-metrics-daemon-422mp" (UID: "366e0b69-56de-45e9-9a8b-87f0a4c8b4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:39.105513 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:39.105434 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret podName:c5ac3838-c267-4741-8925-8c2e90745194 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:40.105424264 +0000 UTC m=+4.059045607 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret") pod "global-pull-secret-syncer-ls7s4" (UID: "c5ac3838-c267-4741-8925-8c2e90745194") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:39.346326 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:39.346280 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6de92e0b_f310_4697_81fc_7293d809946d.slice/crio-6eed1d97244f74ab17fc0f6732f4fc0c1e4c144b44ff672d20b745837df4bea1 WatchSource:0}: Error finding container 6eed1d97244f74ab17fc0f6732f4fc0c1e4c144b44ff672d20b745837df4bea1: Status 404 returned error can't find the container with id 6eed1d97244f74ab17fc0f6732f4fc0c1e4c144b44ff672d20b745837df4bea1 Sep 17 19:06:39.347720 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:39.347500 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27709855_de56_4854_a822_e6044d98d1a4.slice/crio-9dc0cdfb7cad725911275d982c92a76c5e5d096cc67a56c503654d68c361f849 WatchSource:0}: Error finding container 9dc0cdfb7cad725911275d982c92a76c5e5d096cc67a56c503654d68c361f849: Status 404 returned error can't find the container with id 9dc0cdfb7cad725911275d982c92a76c5e5d096cc67a56c503654d68c361f849 Sep 17 19:06:39.351799 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:39.351749 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56c11fe2_25e7_4bb2_b789_ea4178f73d25.slice/crio-e4cace4700e63b7fc42ee6a401f69229fbca2113467b3fa107a12ccfc1ea3ec3 WatchSource:0}: Error finding container e4cace4700e63b7fc42ee6a401f69229fbca2113467b3fa107a12ccfc1ea3ec3: Status 404 returned error can't find the container with id e4cace4700e63b7fc42ee6a401f69229fbca2113467b3fa107a12ccfc1ea3ec3 Sep 17 19:06:39.352207 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:39.352177 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac559d16_e42b_4873_bd8e_abda5c4137fd.slice/crio-d8604d5fa7555eb1da29121e7085379404d85aaa9e31aeae5fd68cafa477ab17 WatchSource:0}: Error finding container d8604d5fa7555eb1da29121e7085379404d85aaa9e31aeae5fd68cafa477ab17: Status 404 returned error can't find the container with id d8604d5fa7555eb1da29121e7085379404d85aaa9e31aeae5fd68cafa477ab17 Sep 17 19:06:39.353563 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:39.353536 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78749d7_d120_4da2_a5fd_29d6bbe672c3.slice/crio-b8a6e21f0355aad83cb58a8f60b330745bfdb585093ecb2ff27d21056e7b420a WatchSource:0}: Error finding container b8a6e21f0355aad83cb58a8f60b330745bfdb585093ecb2ff27d21056e7b420a: Status 404 returned error can't find the container with id b8a6e21f0355aad83cb58a8f60b330745bfdb585093ecb2ff27d21056e7b420a Sep 17 19:06:39.354928 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:39.354912 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode96469d5_497d_4b64_ad08_465aa483e44a.slice/crio-beddf5fb3c5fd3893280d683c69932fff8cfcc75becd20bf41284460356bdb38 WatchSource:0}: Error finding container beddf5fb3c5fd3893280d683c69932fff8cfcc75becd20bf41284460356bdb38: Status 404 returned error can't find the container with id beddf5fb3c5fd3893280d683c69932fff8cfcc75becd20bf41284460356bdb38 Sep 17 19:06:39.355141 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:39.355124 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc439c32d_c42d_4795_8e5e_d4dcf5479e32.slice/crio-7dd95a137747b703cb431d62ed4eecb9199f76657373c5319354aaa7d78b3db6 WatchSource:0}: Error finding container 7dd95a137747b703cb431d62ed4eecb9199f76657373c5319354aaa7d78b3db6: Status 404 returned error can't find the container with id 7dd95a137747b703cb431d62ed4eecb9199f76657373c5319354aaa7d78b3db6 Sep 17 19:06:39.356706 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:39.356677 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bcaefd0_fa6b_4bf6_aa9b_eee050c3626a.slice/crio-b88dc12100809a4c21cf205961e45b8a275ea711cc5ddfc05d5bed87c4bacabb WatchSource:0}: Error finding container b88dc12100809a4c21cf205961e45b8a275ea711cc5ddfc05d5bed87c4bacabb: Status 404 returned error can't find the container with id b88dc12100809a4c21cf205961e45b8a275ea711cc5ddfc05d5bed87c4bacabb Sep 17 19:06:39.357965 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:06:39.357933 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod441acc97_89e4_47a7_9ea1_923c8e7e99a7.slice/crio-dddc6e87120648d7d055c9ac6caf23886258055a82c80375f0abfea535d6dacf WatchSource:0}: Error finding container dddc6e87120648d7d055c9ac6caf23886258055a82c80375f0abfea535d6dacf: Status 404 returned error can't find the container with id dddc6e87120648d7d055c9ac6caf23886258055a82c80375f0abfea535d6dacf Sep 17 19:06:39.460476 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.460431 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 19:01:39 +0000 UTC" deadline="2028-06-23 12:13:34.557571342 +0000 UTC" Sep 17 19:06:39.460476 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.460475 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15473h6m55.097101932s" Sep 17 19:06:39.510269 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.510238 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:39.510423 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:39.510346 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:06:39.519178 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.519146 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-217.ec2.internal" event={"ID":"9f44ab85f08a059c30913c6fa3110b60","Type":"ContainerStarted","Data":"21dc3a4dc6e7ee13b1ef3cfe68f441554c557246fa21d673f061ac73bb56e31d"} Sep 17 19:06:39.521592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.521565 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bnf7d" event={"ID":"441acc97-89e4-47a7-9ea1-923c8e7e99a7","Type":"ContainerStarted","Data":"dddc6e87120648d7d055c9ac6caf23886258055a82c80375f0abfea535d6dacf"} Sep 17 19:06:39.523532 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.523507 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" event={"ID":"c439c32d-c42d-4795-8e5e-d4dcf5479e32","Type":"ContainerStarted","Data":"7dd95a137747b703cb431d62ed4eecb9199f76657373c5319354aaa7d78b3db6"} Sep 17 19:06:39.524896 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.524867 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5w586" event={"ID":"d78749d7-d120-4da2-a5fd-29d6bbe672c3","Type":"ContainerStarted","Data":"b8a6e21f0355aad83cb58a8f60b330745bfdb585093ecb2ff27d21056e7b420a"} Sep 17 19:06:39.526718 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.526699 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ndhbz" event={"ID":"ac559d16-e42b-4873-bd8e-abda5c4137fd","Type":"ContainerStarted","Data":"d8604d5fa7555eb1da29121e7085379404d85aaa9e31aeae5fd68cafa477ab17"} Sep 17 19:06:39.528244 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.528212 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4nvkk" event={"ID":"56c11fe2-25e7-4bb2-b789-ea4178f73d25","Type":"ContainerStarted","Data":"e4cace4700e63b7fc42ee6a401f69229fbca2113467b3fa107a12ccfc1ea3ec3"} Sep 17 19:06:39.529351 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.529332 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-psqk9" event={"ID":"27709855-de56-4854-a822-e6044d98d1a4","Type":"ContainerStarted","Data":"9dc0cdfb7cad725911275d982c92a76c5e5d096cc67a56c503654d68c361f849"} Sep 17 19:06:39.530481 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.530455 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lsmrz" event={"ID":"4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a","Type":"ContainerStarted","Data":"b88dc12100809a4c21cf205961e45b8a275ea711cc5ddfc05d5bed87c4bacabb"} Sep 17 19:06:39.531739 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.531710 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8zwvj" event={"ID":"e96469d5-497d-4b64-ad08-465aa483e44a","Type":"ContainerStarted","Data":"beddf5fb3c5fd3893280d683c69932fff8cfcc75becd20bf41284460356bdb38"} Sep 17 19:06:39.536237 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:39.536205 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" event={"ID":"6de92e0b-f310-4697-81fc-7293d809946d","Type":"ContainerStarted","Data":"6eed1d97244f74ab17fc0f6732f4fc0c1e4c144b44ff672d20b745837df4bea1"} Sep 17 19:06:40.012365 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:40.012323 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v48sq\" (UniqueName: \"kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq\") pod \"network-check-target-t7jj9\" (UID: \"45c9424d-9eba-4232-9e83-85dff9e0132f\") " pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:40.012565 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:40.012545 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:06:40.012633 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:40.012578 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:06:40.012633 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:40.012591 2552 projected.go:196] Error preparing data for projected volume kube-api-access-v48sq for pod openshift-network-diagnostics/network-check-target-t7jj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:40.012735 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:40.012658 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq podName:45c9424d-9eba-4232-9e83-85dff9e0132f nodeName:}" failed. No retries permitted until 2026-09-17 19:06:42.01263857 +0000 UTC m=+5.966259923 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-v48sq" (UniqueName: "kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq") pod "network-check-target-t7jj9" (UID: "45c9424d-9eba-4232-9e83-85dff9e0132f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:40.113428 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:40.113383 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:40.113637 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:40.113476 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:40.113637 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:40.113610 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:40.113763 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:40.113678 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs podName:366e0b69-56de-45e9-9a8b-87f0a4c8b4bf nodeName:}" failed. No retries permitted until 2026-09-17 19:06:42.113656931 +0000 UTC m=+6.067278274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs") pod "network-metrics-daemon-422mp" (UID: "366e0b69-56de-45e9-9a8b-87f0a4c8b4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:40.114150 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:40.114128 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:40.114255 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:40.114189 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret podName:c5ac3838-c267-4741-8925-8c2e90745194 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:42.114173237 +0000 UTC m=+6.067794581 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret") pod "global-pull-secret-syncer-ls7s4" (UID: "c5ac3838-c267-4741-8925-8c2e90745194") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:40.461465 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:40.461395 2552 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 19:01:39 +0000 UTC" deadline="2028-04-25 23:37:07.644599989 +0000 UTC" Sep 17 19:06:40.461465 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:40.461463 2552 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14068h30m27.183140766s" Sep 17 19:06:40.511115 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:40.511073 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:40.511307 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:40.511099 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:40.511307 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:40.511272 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:06:40.511428 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:40.511315 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:06:40.548467 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:40.548424 2552 generic.go:356] "Generic (PLEG): container finished" podID="340d33235ef55881b15f22dfff959742" containerID="505aa13d589d1a108e9dd01759770cbca0b965e9acbab6211f57f59d7d5deb00" exitCode=0 Sep 17 19:06:40.548676 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:40.548524 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" event={"ID":"340d33235ef55881b15f22dfff959742","Type":"ContainerDied","Data":"505aa13d589d1a108e9dd01759770cbca0b965e9acbab6211f57f59d7d5deb00"} Sep 17 19:06:40.579780 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:40.579714 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-217.ec2.internal" podStartSLOduration=2.579688385 podStartE2EDuration="2.579688385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:39.542071653 +0000 UTC m=+3.495693016" watchObservedRunningTime="2026-09-17 19:06:40.579688385 +0000 UTC m=+4.533309750" Sep 17 19:06:41.510424 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:41.510148 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:41.510934 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:41.510555 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:06:41.555382 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:41.555344 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" event={"ID":"340d33235ef55881b15f22dfff959742","Type":"ContainerStarted","Data":"2e1cb368b3fcc1e147258a8bd9dcb3cbf27c7388e1494b10808b728acb70e019"} Sep 17 19:06:42.035192 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:42.035152 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v48sq\" (UniqueName: \"kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq\") pod \"network-check-target-t7jj9\" (UID: \"45c9424d-9eba-4232-9e83-85dff9e0132f\") " pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:42.035381 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:42.035361 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:06:42.035510 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:42.035387 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:06:42.035510 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:42.035430 2552 projected.go:196] Error preparing data for projected volume kube-api-access-v48sq for pod openshift-network-diagnostics/network-check-target-t7jj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:42.035510 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:42.035510 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq podName:45c9424d-9eba-4232-9e83-85dff9e0132f nodeName:}" failed. No retries permitted until 2026-09-17 19:06:46.035488349 +0000 UTC m=+9.989109690 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-v48sq" (UniqueName: "kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq") pod "network-check-target-t7jj9" (UID: "45c9424d-9eba-4232-9e83-85dff9e0132f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:42.136301 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:42.136263 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:42.136481 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:42.136349 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:42.136568 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:42.136542 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:42.136622 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:42.136611 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs podName:366e0b69-56de-45e9-9a8b-87f0a4c8b4bf nodeName:}" failed. No retries permitted until 2026-09-17 19:06:46.136591671 +0000 UTC m=+10.090213022 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs") pod "network-metrics-daemon-422mp" (UID: "366e0b69-56de-45e9-9a8b-87f0a4c8b4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:42.137091 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:42.137069 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:42.137219 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:42.137205 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret podName:c5ac3838-c267-4741-8925-8c2e90745194 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:46.137114167 +0000 UTC m=+10.090735525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret") pod "global-pull-secret-syncer-ls7s4" (UID: "c5ac3838-c267-4741-8925-8c2e90745194") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:42.511122 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:42.511044 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:42.511593 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:42.511170 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:06:42.511593 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:42.511044 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:42.511712 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:42.511643 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:06:43.510507 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:43.510466 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:43.510684 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:43.510606 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:06:44.510439 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:44.510399 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:44.510931 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:44.510537 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:06:44.513109 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:44.513037 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:44.513239 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:44.513197 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:06:45.510526 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:45.510436 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:45.511035 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:45.510582 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:06:46.071477 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:46.071375 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v48sq\" (UniqueName: \"kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq\") pod \"network-check-target-t7jj9\" (UID: \"45c9424d-9eba-4232-9e83-85dff9e0132f\") " pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:46.071662 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:46.071526 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:06:46.071662 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:46.071552 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:06:46.071662 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:46.071565 2552 projected.go:196] Error preparing data for projected volume kube-api-access-v48sq for pod openshift-network-diagnostics/network-check-target-t7jj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:46.071662 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:46.071624 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq podName:45c9424d-9eba-4232-9e83-85dff9e0132f nodeName:}" failed. No retries permitted until 2026-09-17 19:06:54.071605829 +0000 UTC m=+18.025227182 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-v48sq" (UniqueName: "kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq") pod "network-check-target-t7jj9" (UID: "45c9424d-9eba-4232-9e83-85dff9e0132f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:46.172114 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:46.172071 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:46.172294 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:46.172145 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:46.172294 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:46.172212 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:46.172294 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:46.172275 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:46.172294 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:46.172280 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs podName:366e0b69-56de-45e9-9a8b-87f0a4c8b4bf nodeName:}" failed. No retries permitted until 2026-09-17 19:06:54.172260674 +0000 UTC m=+18.125882048 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs") pod "network-metrics-daemon-422mp" (UID: "366e0b69-56de-45e9-9a8b-87f0a4c8b4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:46.172570 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:46.172309 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret podName:c5ac3838-c267-4741-8925-8c2e90745194 nodeName:}" failed. No retries permitted until 2026-09-17 19:06:54.172299658 +0000 UTC m=+18.125920997 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret") pod "global-pull-secret-syncer-ls7s4" (UID: "c5ac3838-c267-4741-8925-8c2e90745194") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:46.511453 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:46.511374 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:46.511916 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:46.511511 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:06:46.511916 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:46.511818 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:46.511916 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:46.511885 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:06:47.510863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:47.510399 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:47.510863 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:47.510514 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:06:48.510884 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:48.510847 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:48.511333 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:48.510895 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:48.511333 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:48.511019 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:06:48.511333 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:48.511108 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:06:49.510918 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:49.510882 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:49.511403 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:49.511002 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:06:50.510545 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:50.510524 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:50.510631 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:50.510617 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:06:50.510710 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:50.510694 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:50.510802 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:50.510783 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:06:51.510196 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:51.510160 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:51.510654 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:51.510281 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:06:52.510369 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:52.510332 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:52.510814 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:52.510469 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:06:52.510814 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:52.510503 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:52.510814 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:52.510618 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:06:53.510646 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:53.510614 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:53.511121 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:53.510727 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:06:54.131366 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:54.131339 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v48sq\" (UniqueName: \"kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq\") pod \"network-check-target-t7jj9\" (UID: \"45c9424d-9eba-4232-9e83-85dff9e0132f\") " pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:54.131531 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:54.131462 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:06:54.131531 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:54.131475 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:06:54.131531 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:54.131484 2552 projected.go:196] Error preparing data for projected volume kube-api-access-v48sq for pod openshift-network-diagnostics/network-check-target-t7jj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:54.131531 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:54.131528 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq podName:45c9424d-9eba-4232-9e83-85dff9e0132f nodeName:}" failed. No retries permitted until 2026-09-17 19:07:10.131515939 +0000 UTC m=+34.085137276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-v48sq" (UniqueName: "kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq") pod "network-check-target-t7jj9" (UID: "45c9424d-9eba-4232-9e83-85dff9e0132f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:06:54.232668 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:54.232637 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:54.232842 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:54.232685 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:54.232842 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:54.232793 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:54.232842 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:54.232795 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:54.232998 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:54.232847 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret podName:c5ac3838-c267-4741-8925-8c2e90745194 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:10.232834744 +0000 UTC m=+34.186456080 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret") pod "global-pull-secret-syncer-ls7s4" (UID: "c5ac3838-c267-4741-8925-8c2e90745194") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:06:54.232998 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:54.232866 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs podName:366e0b69-56de-45e9-9a8b-87f0a4c8b4bf nodeName:}" failed. No retries permitted until 2026-09-17 19:07:10.232855618 +0000 UTC m=+34.186476954 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs") pod "network-metrics-daemon-422mp" (UID: "366e0b69-56de-45e9-9a8b-87f0a4c8b4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:06:54.510384 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:54.510310 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:54.510536 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:54.510310 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:54.510536 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:54.510440 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:06:54.510536 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:54.510487 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:06:55.510396 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:55.510363 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:55.510819 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:55.510485 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:06:56.511524 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:56.511487 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:56.511993 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:56.511602 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:06:56.511993 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:56.511654 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:56.511993 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:56.511742 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:06:57.510190 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:57.510033 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:57.510190 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:57.510152 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:06:58.510521 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.510348 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:06:58.511277 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.510421 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:06:58.511277 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:58.510687 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:06:58.511277 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:58.510587 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:06:58.592193 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.592163 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" event={"ID":"c439c32d-c42d-4795-8e5e-d4dcf5479e32","Type":"ContainerStarted","Data":"7da9ef4c711d0c9c14ab6e5fcd89c8890078feb9f5498d672df196363bf290f1"} Sep 17 19:06:58.592341 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.592239 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" event={"ID":"c439c32d-c42d-4795-8e5e-d4dcf5479e32","Type":"ContainerStarted","Data":"5bc141484ac9594bf2b6871ff52dc8de2baee8dc98ccb2d0517c58714cc719a3"} Sep 17 19:06:58.592341 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.592269 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" event={"ID":"c439c32d-c42d-4795-8e5e-d4dcf5479e32","Type":"ContainerStarted","Data":"e528ada0534ec65c2a95fa103152131122c9e58462bbb54a6c5a356ec9de9c5a"} Sep 17 19:06:58.592341 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.592303 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" event={"ID":"c439c32d-c42d-4795-8e5e-d4dcf5479e32","Type":"ContainerStarted","Data":"88fe07ea6ec0cf728dc51a89b6e402f72a8fac0d088fb7011d2ba5341bf5fbac"} Sep 17 19:06:58.592341 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.592333 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" event={"ID":"c439c32d-c42d-4795-8e5e-d4dcf5479e32","Type":"ContainerStarted","Data":"db7ff77702011d80370341e6344e869553ffd8de92d90698edb43c5c53e78068"} Sep 17 19:06:58.592568 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.592363 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" event={"ID":"c439c32d-c42d-4795-8e5e-d4dcf5479e32","Type":"ContainerStarted","Data":"f24552f808f04e52c5c0e6b2a2f89e46661577863f1d30101a69f359d4ab99e7"} Sep 17 19:06:58.593530 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.593508 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5w586" event={"ID":"d78749d7-d120-4da2-a5fd-29d6bbe672c3","Type":"ContainerStarted","Data":"f81a02a317318939b4f1c8f8425490eaeaebc3804ac7c41d1f4425ad6d4d9a9a"} Sep 17 19:06:58.594917 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.594875 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ndhbz" event={"ID":"ac559d16-e42b-4873-bd8e-abda5c4137fd","Type":"ContainerStarted","Data":"6ffb9a6a0f4391fdd36cc919d60edaff0bc6b3f878b3ab6731627cb4444aedb7"} Sep 17 19:06:58.596206 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.596189 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4nvkk" event={"ID":"56c11fe2-25e7-4bb2-b789-ea4178f73d25","Type":"ContainerStarted","Data":"345ecac4a3065d8847e55b3846d672ccb52191095673ec13b83e34c1b620ca86"} Sep 17 19:06:58.597412 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.597393 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-psqk9" event={"ID":"27709855-de56-4854-a822-e6044d98d1a4","Type":"ContainerStarted","Data":"0ad35e02f2e9b0032296f875b6ab370c785692978976a61cd6a720664b199a37"} Sep 17 19:06:58.598583 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.598564 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lsmrz" event={"ID":"4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a","Type":"ContainerStarted","Data":"764344023b6be1c917e0c8ce3cdb7e4baec95d88c0b52ae396cfa93db8490ac6"} Sep 17 19:06:58.599982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.599961 2552 generic.go:356] "Generic (PLEG): container finished" podID="e96469d5-497d-4b64-ad08-465aa483e44a" containerID="9f7ca97709841af6d9685fb0955e1ff994b33739a2639c847efa5665c23d89fd" exitCode=0 Sep 17 19:06:58.600051 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.600031 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8zwvj" event={"ID":"e96469d5-497d-4b64-ad08-465aa483e44a","Type":"ContainerDied","Data":"9f7ca97709841af6d9685fb0955e1ff994b33739a2639c847efa5665c23d89fd"} Sep 17 19:06:58.601353 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.601333 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" event={"ID":"6de92e0b-f310-4697-81fc-7293d809946d","Type":"ContainerStarted","Data":"8e0cbfbcb03f26a150274141449ad6455c1b8ea9193907b74c53a4979e6ddc95"} Sep 17 19:06:58.627699 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.627654 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-217.ec2.internal" podStartSLOduration=20.627642908 podStartE2EDuration="20.627642908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:41.5746663 +0000 UTC m=+5.528287661" watchObservedRunningTime="2026-09-17 19:06:58.627642908 +0000 UTC m=+22.581264264" Sep 17 19:06:58.628165 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.628144 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5w586" podStartSLOduration=6.836261765 podStartE2EDuration="21.628137637s" totalImagesPullingTime="14.791875872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:58.627549667 +0000 UTC m=+22.581171037" watchObservedRunningTime="2026-09-17 19:06:58.628137637 +0000 UTC m=+22.581758994" Sep 17 19:06:58.697816 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.697766 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4nvkk" podStartSLOduration=3.555932995 podStartE2EDuration="21.697745545s" totalImagesPullingTime="18.14181255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:58.665277151 +0000 UTC m=+22.618898509" watchObservedRunningTime="2026-09-17 19:06:58.697745545 +0000 UTC m=+22.651366904" Sep 17 19:06:58.748226 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.748042 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lsmrz" podStartSLOduration=3.663671581 podStartE2EDuration="21.748026816s" totalImagesPullingTime="18.084355235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:58.698333872 +0000 UTC m=+22.651955231" watchObservedRunningTime="2026-09-17 19:06:58.748026816 +0000 UTC m=+22.701648174" Sep 17 19:06:58.748501 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.748474 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-psqk9" podStartSLOduration=3.615124438 podStartE2EDuration="21.748463334s" totalImagesPullingTime="18.133338896s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:58.747811636 +0000 UTC m=+22.701432993" watchObservedRunningTime="2026-09-17 19:06:58.748463334 +0000 UTC m=+22.702084691" Sep 17 19:06:58.819717 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.819674 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ndhbz" podStartSLOduration=3.73074342 podStartE2EDuration="21.819657916s" totalImagesPullingTime="18.088914496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:58.781036259 +0000 UTC m=+22.734657616" watchObservedRunningTime="2026-09-17 19:06:58.819657916 +0000 UTC m=+22.773279265" Sep 17 19:06:58.826008 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:58.825986 2552 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 17 19:06:59.257401 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:59.257330 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5w586" Sep 17 19:06:59.257840 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:59.257823 2552 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5w586" Sep 17 19:06:59.466269 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:59.466135 2552 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T19:06:58.826003952Z","UUID":"4ec7d9e8-63cd-4792-8aed-ddf16593e59a","Handler":null,"Name":"","Endpoint":""} Sep 17 19:06:59.467777 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:59.467756 2552 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 17 19:06:59.467926 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:59.467788 2552 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 17 19:06:59.510286 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:59.510222 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:06:59.510406 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:06:59.510332 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:06:59.605004 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:59.604963 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bnf7d" event={"ID":"441acc97-89e4-47a7-9ea1-923c8e7e99a7","Type":"ContainerStarted","Data":"00384a9aa24dd0c5b46a5e2912e78bf26678ad2d68e583dfb0eea95d2c966f04"} Sep 17 19:06:59.608922 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:59.608875 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" event={"ID":"6de92e0b-f310-4697-81fc-7293d809946d","Type":"ContainerStarted","Data":"1c30810664283e9885d610da79cb80240e5b7fe223cec5ba93c9208a5a164f21"} Sep 17 19:06:59.609845 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:59.609829 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5w586" Sep 17 19:06:59.610363 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:59.610344 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5w586" Sep 17 19:06:59.631460 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:06:59.631421 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bnf7d" podStartSLOduration=4.543311532 podStartE2EDuration="22.631402369s" totalImagesPullingTime="18.088090837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:06:59.630963835 +0000 UTC m=+23.584585193" watchObservedRunningTime="2026-09-17 19:06:59.631402369 +0000 UTC m=+23.585023728" Sep 17 19:07:00.510970 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:00.510925 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:00.510970 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:00.510944 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:07:00.511214 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:00.511050 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:07:00.511214 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:00.511164 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:07:00.615082 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:00.614999 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" event={"ID":"c439c32d-c42d-4795-8e5e-d4dcf5479e32","Type":"ContainerStarted","Data":"f74aa02c1ad2bef279921b26d44f1a8c1b78b7f0a47a5451993e56c86a52457c"} Sep 17 19:07:00.617516 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:00.617487 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" event={"ID":"6de92e0b-f310-4697-81fc-7293d809946d","Type":"ContainerStarted","Data":"7723eb36ab8c7375f87e13812f7731b0487c73e0c044d71a9419d434a80050ba"} Sep 17 19:07:01.510230 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:01.510194 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:07:01.510424 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:01.510309 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:07:02.510704 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:02.510675 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:02.511119 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:02.510810 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:07:02.511119 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:02.510860 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:07:02.511119 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:02.510980 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:07:03.510731 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:03.510510 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:07:03.511237 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:03.510757 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:07:03.628155 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:03.628121 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" event={"ID":"c439c32d-c42d-4795-8e5e-d4dcf5479e32","Type":"ContainerStarted","Data":"ae4a5ed41a4f9581b02e5ad8db2e447102dbf436898a787151989973c79e6cf9"} Sep 17 19:07:03.628355 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:03.628334 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:07:03.630075 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:03.630052 2552 generic.go:356] "Generic (PLEG): container finished" podID="e96469d5-497d-4b64-ad08-465aa483e44a" containerID="adb21115da4bc912871b68f0574e8dcbeda15deb62c62609f34c91d88ae001c7" exitCode=0 Sep 17 19:07:03.630174 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:03.630101 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8zwvj" event={"ID":"e96469d5-497d-4b64-ad08-465aa483e44a","Type":"ContainerDied","Data":"adb21115da4bc912871b68f0574e8dcbeda15deb62c62609f34c91d88ae001c7"} Sep 17 19:07:03.643917 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:03.643877 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:07:03.664188 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:03.664141 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" podStartSLOduration=8.519435183 podStartE2EDuration="26.664127417s" totalImagesPullingTime="18.144692234s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:03.662862085 +0000 UTC m=+27.616483444" watchObservedRunningTime="2026-09-17 19:07:03.664127417 +0000 UTC m=+27.617748774" Sep 17 19:07:03.664392 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:03.664366 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-df828" podStartSLOduration=6.472453293 podStartE2EDuration="26.664358968s" totalImagesPullingTime="20.191905675s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:00.653089462 +0000 UTC m=+24.606710816" watchObservedRunningTime="2026-09-17 19:07:03.664358968 +0000 UTC m=+27.617980328" Sep 17 19:07:04.513292 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:04.513260 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:07:04.513767 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:04.513355 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:07:04.513767 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:04.513381 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:04.513767 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:04.513422 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:07:04.635428 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:04.635402 2552 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 17 19:07:04.635977 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:04.635953 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:07:04.659350 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:04.659322 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:07:04.885893 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:04.885710 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t7jj9"] Sep 17 19:07:04.886091 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:04.885977 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:04.886091 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:04.886071 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:07:04.891741 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:04.891717 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ls7s4"] Sep 17 19:07:04.891886 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:04.891832 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:07:04.892017 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:04.891992 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:07:04.892416 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:04.892394 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-422mp"] Sep 17 19:07:04.892509 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:04.892484 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:07:04.892599 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:04.892582 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:07:05.239880 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:05.239849 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:07:05.639436 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:05.639395 2552 generic.go:356] "Generic (PLEG): container finished" podID="e96469d5-497d-4b64-ad08-465aa483e44a" containerID="aeb0737359afe1b7ed70c43a7e4eb1db5dc77c4b863174d429a961351d0b83da" exitCode=0 Sep 17 19:07:05.639927 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:05.639484 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8zwvj" event={"ID":"e96469d5-497d-4b64-ad08-465aa483e44a","Type":"ContainerDied","Data":"aeb0737359afe1b7ed70c43a7e4eb1db5dc77c4b863174d429a961351d0b83da"} Sep 17 19:07:06.511101 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:06.511066 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:06.511282 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:06.511168 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:07:06.511282 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:06.511193 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:07:06.511394 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:06.511293 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:07:06.511394 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:06.511321 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:07:06.511394 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:06.511383 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:07:07.646392 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:07.646358 2552 generic.go:356] "Generic (PLEG): container finished" podID="e96469d5-497d-4b64-ad08-465aa483e44a" containerID="7c2935a322529955b60b74d359451e665beb5a8fda77487fbe2260b0e96f0f49" exitCode=0 Sep 17 19:07:07.646804 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:07.646407 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8zwvj" event={"ID":"e96469d5-497d-4b64-ad08-465aa483e44a","Type":"ContainerDied","Data":"7c2935a322529955b60b74d359451e665beb5a8fda77487fbe2260b0e96f0f49"} Sep 17 19:07:08.062591 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:08.062548 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:08.161610 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:08.161572 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:08.510627 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:08.510546 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:07:08.510627 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:08.510582 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:07:08.510627 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:08.510546 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:08.510891 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:08.510688 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:07:08.510891 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:08.510773 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:07:08.510891 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:08.510856 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:07:09.647447 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:09.647415 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:09.752054 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:09.752016 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:09.867223 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:09.867186 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:10.148918 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:10.148871 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v48sq\" (UniqueName: \"kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq\") pod \"network-check-target-t7jj9\" (UID: \"45c9424d-9eba-4232-9e83-85dff9e0132f\") " pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:10.149089 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:10.149035 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 19:07:10.149089 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:10.149052 2552 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 19:07:10.149089 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:10.149061 2552 projected.go:196] Error preparing data for projected volume kube-api-access-v48sq for pod openshift-network-diagnostics/network-check-target-t7jj9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:07:10.149220 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:10.149117 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq podName:45c9424d-9eba-4232-9e83-85dff9e0132f nodeName:}" failed. No retries permitted until 2026-09-17 19:07:42.14910233 +0000 UTC m=+66.102723666 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-v48sq" (UniqueName: "kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq") pod "network-check-target-t7jj9" (UID: "45c9424d-9eba-4232-9e83-85dff9e0132f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 19:07:10.217194 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:10.217156 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:10.250216 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:10.250175 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:07:10.250397 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:10.250253 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:07:10.250397 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:10.250343 2552 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 19:07:10.250397 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:10.250371 2552 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:07:10.250549 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:10.250419 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret podName:c5ac3838-c267-4741-8925-8c2e90745194 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:42.250397952 +0000 UTC m=+66.204019295 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret") pod "global-pull-secret-syncer-ls7s4" (UID: "c5ac3838-c267-4741-8925-8c2e90745194") : object "kube-system"/"original-pull-secret" not registered Sep 17 19:07:10.250549 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:10.250441 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs podName:366e0b69-56de-45e9-9a8b-87f0a4c8b4bf nodeName:}" failed. No retries permitted until 2026-09-17 19:07:42.250429862 +0000 UTC m=+66.204051204 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs") pod "network-metrics-daemon-422mp" (UID: "366e0b69-56de-45e9-9a8b-87f0a4c8b4bf") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 19:07:10.511315 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:10.510494 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:10.511315 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:10.510530 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:07:10.511315 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:10.510635 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-t7jj9" podUID="45c9424d-9eba-4232-9e83-85dff9e0132f" Sep 17 19:07:10.511315 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:10.511048 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ls7s4" podUID="c5ac3838-c267-4741-8925-8c2e90745194" Sep 17 19:07:10.511315 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:10.511142 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:07:10.511315 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:10.511226 2552 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-422mp" podUID="366e0b69-56de-45e9-9a8b-87f0a4c8b4bf" Sep 17 19:07:10.878238 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:10.878206 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:11.290487 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.290460 2552 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-217.ec2.internal" event="NodeReady" Sep 17 19:07:11.290665 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.290611 2552 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 17 19:07:11.374461 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.374428 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5bxxw"] Sep 17 19:07:11.379054 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.379031 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-95lc8"] Sep 17 19:07:11.379181 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.379166 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:11.381890 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.381831 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 17 19:07:11.382158 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.382137 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-95lc8" Sep 17 19:07:11.382264 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.382161 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dpmzt\"" Sep 17 19:07:11.382870 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.382853 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 17 19:07:11.387030 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.386746 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 17 19:07:11.387030 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.386774 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 17 19:07:11.387030 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.386798 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4xslp\"" Sep 17 19:07:11.387030 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.386891 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 17 19:07:11.394955 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.394879 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5bxxw"] Sep 17 19:07:11.411434 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.411406 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-95lc8"] Sep 17 19:07:11.458052 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.458021 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ngb9\" (UniqueName: \"kubernetes.io/projected/bce04825-89fe-47fa-a72f-0a24aa333a75-kube-api-access-2ngb9\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:11.458226 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.458107 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:11.458226 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.458144 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bce04825-89fe-47fa-a72f-0a24aa333a75-tmp-dir\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:11.458226 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.458188 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bce04825-89fe-47fa-a72f-0a24aa333a75-config-volume\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:11.559318 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.559232 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bce04825-89fe-47fa-a72f-0a24aa333a75-tmp-dir\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:11.559318 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.559271 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwvhk\" (UniqueName: \"kubernetes.io/projected/36e00d1e-728d-4377-bca4-5f14d2a88b7e-kube-api-access-cwvhk\") pod \"ingress-canary-95lc8\" (UID: \"36e00d1e-728d-4377-bca4-5f14d2a88b7e\") " pod="openshift-ingress-canary/ingress-canary-95lc8" Sep 17 19:07:11.559318 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.559317 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bce04825-89fe-47fa-a72f-0a24aa333a75-config-volume\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:11.559577 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.559349 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ngb9\" (UniqueName: \"kubernetes.io/projected/bce04825-89fe-47fa-a72f-0a24aa333a75-kube-api-access-2ngb9\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:11.559577 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.559372 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36e00d1e-728d-4377-bca4-5f14d2a88b7e-cert\") pod \"ingress-canary-95lc8\" (UID: \"36e00d1e-728d-4377-bca4-5f14d2a88b7e\") " pod="openshift-ingress-canary/ingress-canary-95lc8" Sep 17 19:07:11.559577 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.559485 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:11.559700 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:11.559642 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 19:07:11.559744 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.559699 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bce04825-89fe-47fa-a72f-0a24aa333a75-tmp-dir\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:11.559744 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:11.559719 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls podName:bce04825-89fe-47fa-a72f-0a24aa333a75 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:12.059701629 +0000 UTC m=+36.013322972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls") pod "dns-default-5bxxw" (UID: "bce04825-89fe-47fa-a72f-0a24aa333a75") : secret "dns-default-metrics-tls" not found Sep 17 19:07:11.559934 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.559895 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bce04825-89fe-47fa-a72f-0a24aa333a75-config-volume\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:11.570971 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.570830 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ngb9\" (UniqueName: \"kubernetes.io/projected/bce04825-89fe-47fa-a72f-0a24aa333a75-kube-api-access-2ngb9\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:11.660123 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.660096 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36e00d1e-728d-4377-bca4-5f14d2a88b7e-cert\") pod \"ingress-canary-95lc8\" (UID: \"36e00d1e-728d-4377-bca4-5f14d2a88b7e\") " pod="openshift-ingress-canary/ingress-canary-95lc8" Sep 17 19:07:11.660290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.660187 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwvhk\" (UniqueName: \"kubernetes.io/projected/36e00d1e-728d-4377-bca4-5f14d2a88b7e-kube-api-access-cwvhk\") pod \"ingress-canary-95lc8\" (UID: \"36e00d1e-728d-4377-bca4-5f14d2a88b7e\") " pod="openshift-ingress-canary/ingress-canary-95lc8" Sep 17 19:07:11.663009 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.662981 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36e00d1e-728d-4377-bca4-5f14d2a88b7e-cert\") pod \"ingress-canary-95lc8\" (UID: \"36e00d1e-728d-4377-bca4-5f14d2a88b7e\") " pod="openshift-ingress-canary/ingress-canary-95lc8" Sep 17 19:07:11.670667 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.670644 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwvhk\" (UniqueName: \"kubernetes.io/projected/36e00d1e-728d-4377-bca4-5f14d2a88b7e-kube-api-access-cwvhk\") pod \"ingress-canary-95lc8\" (UID: \"36e00d1e-728d-4377-bca4-5f14d2a88b7e\") " pod="openshift-ingress-canary/ingress-canary-95lc8" Sep 17 19:07:11.697671 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:11.697641 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-95lc8" Sep 17 19:07:12.062992 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.062959 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:12.063753 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:12.063099 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 19:07:12.063753 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:12.063168 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls podName:bce04825-89fe-47fa-a72f-0a24aa333a75 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:13.063149748 +0000 UTC m=+37.016771101 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls") pod "dns-default-5bxxw" (UID: "bce04825-89fe-47fa-a72f-0a24aa333a75") : secret "dns-default-metrics-tls" not found Sep 17 19:07:12.102525 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.102443 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-95lc8"] Sep 17 19:07:12.510579 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.510496 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:07:12.510719 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.510499 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:07:12.510719 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.510499 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:12.514028 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.514002 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 19:07:12.514144 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.514053 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 19:07:12.515085 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.515069 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b9ftw\"" Sep 17 19:07:12.515528 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.515516 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 19:07:12.523321 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.523298 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 19:07:12.523663 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.523650 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-47zq2\"" Sep 17 19:07:12.646430 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.646395 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd"] Sep 17 19:07:12.649825 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.649802 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" Sep 17 19:07:12.658186 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.658140 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-x2kmh\"" Sep 17 19:07:12.658304 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.658211 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 17 19:07:12.658304 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.658163 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 17 19:07:12.658527 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.658509 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 17 19:07:12.658527 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.658509 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 17 19:07:12.663396 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.663373 2552 generic.go:356] "Generic (PLEG): container finished" podID="e96469d5-497d-4b64-ad08-465aa483e44a" containerID="ff2ab73623c3ab82a5d5616edcac63b86eb607081b08a59fd50961f5c9341707" exitCode=0 Sep 17 19:07:12.663511 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.663445 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8zwvj" event={"ID":"e96469d5-497d-4b64-ad08-465aa483e44a","Type":"ContainerDied","Data":"ff2ab73623c3ab82a5d5616edcac63b86eb607081b08a59fd50961f5c9341707"} Sep 17 19:07:12.664742 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.664711 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-95lc8" event={"ID":"36e00d1e-728d-4377-bca4-5f14d2a88b7e","Type":"ContainerStarted","Data":"52f75f9b7b9b381b9593b327177338642770b133f3b6f0aca5de4598cdc7f606"} Sep 17 19:07:12.665703 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.665284 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:12.668097 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.668078 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd"] Sep 17 19:07:12.766372 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.766347 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:12.766497 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.766468 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/33012019-3ef9-475a-a6ab-f7e445f79ea9-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-zsjnd\" (UID: \"33012019-3ef9-475a-a6ab-f7e445f79ea9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" Sep 17 19:07:12.766626 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.766524 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/33012019-3ef9-475a-a6ab-f7e445f79ea9-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-zsjnd\" (UID: \"33012019-3ef9-475a-a6ab-f7e445f79ea9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" Sep 17 19:07:12.766626 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.766555 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92nbr\" (UniqueName: \"kubernetes.io/projected/33012019-3ef9-475a-a6ab-f7e445f79ea9-kube-api-access-92nbr\") pod \"kube-storage-version-migrator-operator-55889f7b47-zsjnd\" (UID: \"33012019-3ef9-475a-a6ab-f7e445f79ea9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" Sep 17 19:07:12.867374 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.867341 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/33012019-3ef9-475a-a6ab-f7e445f79ea9-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-zsjnd\" (UID: \"33012019-3ef9-475a-a6ab-f7e445f79ea9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" Sep 17 19:07:12.867560 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.867393 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/33012019-3ef9-475a-a6ab-f7e445f79ea9-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-zsjnd\" (UID: \"33012019-3ef9-475a-a6ab-f7e445f79ea9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" Sep 17 19:07:12.867560 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.867414 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92nbr\" (UniqueName: \"kubernetes.io/projected/33012019-3ef9-475a-a6ab-f7e445f79ea9-kube-api-access-92nbr\") pod \"kube-storage-version-migrator-operator-55889f7b47-zsjnd\" (UID: \"33012019-3ef9-475a-a6ab-f7e445f79ea9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" Sep 17 19:07:12.868083 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.868055 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/33012019-3ef9-475a-a6ab-f7e445f79ea9-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-zsjnd\" (UID: \"33012019-3ef9-475a-a6ab-f7e445f79ea9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" Sep 17 19:07:12.870521 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.870478 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/33012019-3ef9-475a-a6ab-f7e445f79ea9-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-zsjnd\" (UID: \"33012019-3ef9-475a-a6ab-f7e445f79ea9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" Sep 17 19:07:12.879299 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.879272 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92nbr\" (UniqueName: \"kubernetes.io/projected/33012019-3ef9-475a-a6ab-f7e445f79ea9-kube-api-access-92nbr\") pod \"kube-storage-version-migrator-operator-55889f7b47-zsjnd\" (UID: \"33012019-3ef9-475a-a6ab-f7e445f79ea9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" Sep 17 19:07:12.961147 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:12.961104 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" Sep 17 19:07:13.068549 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.068513 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:13.069060 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:13.068649 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 19:07:13.069060 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:13.068714 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls podName:bce04825-89fe-47fa-a72f-0a24aa333a75 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:15.068697082 +0000 UTC m=+39.022318424 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls") pod "dns-default-5bxxw" (UID: "bce04825-89fe-47fa-a72f-0a24aa333a75") : secret "dns-default-metrics-tls" not found Sep 17 19:07:13.246558 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.246270 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd"] Sep 17 19:07:13.250495 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:07:13.250466 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33012019_3ef9_475a_a6ab_f7e445f79ea9.slice/crio-74e653ff509d070b52a32b9d39f9838b47f90501fdbfb1ebb8bf7d8e08d572ee WatchSource:0}: Error finding container 74e653ff509d070b52a32b9d39f9838b47f90501fdbfb1ebb8bf7d8e08d572ee: Status 404 returned error can't find the container with id 74e653ff509d070b52a32b9d39f9838b47f90501fdbfb1ebb8bf7d8e08d572ee Sep 17 19:07:13.671484 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.671447 2552 generic.go:356] "Generic (PLEG): container finished" podID="e96469d5-497d-4b64-ad08-465aa483e44a" containerID="bbcabd8c942482231d4df6f3bb85c0fdb5c90b98ede432152e558b6d3b3164af" exitCode=0 Sep 17 19:07:13.671687 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.671543 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8zwvj" event={"ID":"e96469d5-497d-4b64-ad08-465aa483e44a","Type":"ContainerDied","Data":"bbcabd8c942482231d4df6f3bb85c0fdb5c90b98ede432152e558b6d3b3164af"} Sep 17 19:07:13.672952 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.672927 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" event={"ID":"33012019-3ef9-475a-a6ab-f7e445f79ea9","Type":"ContainerStarted","Data":"74e653ff509d070b52a32b9d39f9838b47f90501fdbfb1ebb8bf7d8e08d572ee"} Sep 17 19:07:13.842451 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.842413 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hlh59"] Sep 17 19:07:13.851515 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.851488 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:13.857513 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.857487 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 17 19:07:13.858555 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.858025 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r9nrg\"" Sep 17 19:07:13.858555 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.858208 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 17 19:07:13.858555 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.858355 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 17 19:07:13.858748 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.858601 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 17 19:07:13.869427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.869403 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hlh59"] Sep 17 19:07:13.871682 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.871659 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:13.980969 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.980863 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5404dd23-6e97-439f-a95d-8b3c8e533de8-data-volume\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:13.980969 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.980932 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5404dd23-6e97-439f-a95d-8b3c8e533de8-crio-socket\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:13.980969 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.980963 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m67d\" (UniqueName: \"kubernetes.io/projected/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-api-access-6m67d\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:13.981206 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.981035 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:13.981206 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:13.981065 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:14.081657 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.081615 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:14.082149 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.081734 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5404dd23-6e97-439f-a95d-8b3c8e533de8-data-volume\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:14.082149 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.081760 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5404dd23-6e97-439f-a95d-8b3c8e533de8-crio-socket\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:14.082149 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:14.081764 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 19:07:14.082149 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.081783 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m67d\" (UniqueName: \"kubernetes.io/projected/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-api-access-6m67d\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:14.082149 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:14.081860 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm podName:5404dd23-6e97-439f-a95d-8b3c8e533de8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:14.581839733 +0000 UTC m=+38.535461087 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hlh59" (UID: "5404dd23-6e97-439f-a95d-8b3c8e533de8") : configmap "kube-rbac-proxy" not found Sep 17 19:07:14.082149 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.081884 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:14.082149 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:14.082004 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 19:07:14.082149 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:14.082040 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls podName:5404dd23-6e97-439f-a95d-8b3c8e533de8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:14.582029124 +0000 UTC m=+38.535650474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hlh59" (UID: "5404dd23-6e97-439f-a95d-8b3c8e533de8") : secret "insights-runtime-extractor-tls" not found Sep 17 19:07:14.082149 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.082071 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5404dd23-6e97-439f-a95d-8b3c8e533de8-crio-socket\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:14.082564 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.082179 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5404dd23-6e97-439f-a95d-8b3c8e533de8-data-volume\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:14.091918 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.091871 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m67d\" (UniqueName: \"kubernetes.io/projected/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-api-access-6m67d\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:14.586278 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.586242 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:14.586407 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.586352 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:14.586407 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:14.586358 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 19:07:14.586470 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:14.586417 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm podName:5404dd23-6e97-439f-a95d-8b3c8e533de8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:15.58640051 +0000 UTC m=+39.540021863 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hlh59" (UID: "5404dd23-6e97-439f-a95d-8b3c8e533de8") : configmap "kube-rbac-proxy" not found Sep 17 19:07:14.586470 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:14.586448 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 19:07:14.586541 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:14.586489 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls podName:5404dd23-6e97-439f-a95d-8b3c8e533de8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:15.586477051 +0000 UTC m=+39.540098387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hlh59" (UID: "5404dd23-6e97-439f-a95d-8b3c8e533de8") : secret "insights-runtime-extractor-tls" not found Sep 17 19:07:14.637160 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.637127 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr"] Sep 17 19:07:14.652005 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.651983 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" Sep 17 19:07:14.655090 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.655027 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr"] Sep 17 19:07:14.659156 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.658802 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 17 19:07:14.659156 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.659138 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wh9lc\"" Sep 17 19:07:14.659435 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.659414 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 17 19:07:14.659938 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.659626 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 17 19:07:14.672095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.672070 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 17 19:07:14.680624 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.680592 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8zwvj" event={"ID":"e96469d5-497d-4b64-ad08-465aa483e44a","Type":"ContainerStarted","Data":"55ac2348aea7243635269138744fa85b501846893291176c7ece54e9c4ce6e92"} Sep 17 19:07:14.715058 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.715007 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8zwvj" podStartSLOduration=8.934121453 podStartE2EDuration="37.714986094s" totalImagesPullingTime="28.780864641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:14.713296547 +0000 UTC m=+38.666917907" watchObservedRunningTime="2026-09-17 19:07:14.714986094 +0000 UTC m=+38.668607455" Sep 17 19:07:14.787290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.787258 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6d3b5566-1d04-4bd1-98eb-e69ace161cd3-config\") pod \"service-ca-operator-57cdc4b768-glcdr\" (UID: \"6d3b5566-1d04-4bd1-98eb-e69ace161cd3\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" Sep 17 19:07:14.787290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.787289 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gtgc\" (UniqueName: \"kubernetes.io/projected/6d3b5566-1d04-4bd1-98eb-e69ace161cd3-kube-api-access-2gtgc\") pod \"service-ca-operator-57cdc4b768-glcdr\" (UID: \"6d3b5566-1d04-4bd1-98eb-e69ace161cd3\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" Sep 17 19:07:14.787502 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.787425 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6d3b5566-1d04-4bd1-98eb-e69ace161cd3-serving-cert\") pod \"service-ca-operator-57cdc4b768-glcdr\" (UID: \"6d3b5566-1d04-4bd1-98eb-e69ace161cd3\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" Sep 17 19:07:14.856513 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.856431 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:14.888488 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.888451 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6d3b5566-1d04-4bd1-98eb-e69ace161cd3-serving-cert\") pod \"service-ca-operator-57cdc4b768-glcdr\" (UID: \"6d3b5566-1d04-4bd1-98eb-e69ace161cd3\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" Sep 17 19:07:14.888655 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.888562 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6d3b5566-1d04-4bd1-98eb-e69ace161cd3-config\") pod \"service-ca-operator-57cdc4b768-glcdr\" (UID: \"6d3b5566-1d04-4bd1-98eb-e69ace161cd3\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" Sep 17 19:07:14.888655 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.888591 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gtgc\" (UniqueName: \"kubernetes.io/projected/6d3b5566-1d04-4bd1-98eb-e69ace161cd3-kube-api-access-2gtgc\") pod \"service-ca-operator-57cdc4b768-glcdr\" (UID: \"6d3b5566-1d04-4bd1-98eb-e69ace161cd3\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" Sep 17 19:07:14.889149 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.889126 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6d3b5566-1d04-4bd1-98eb-e69ace161cd3-config\") pod \"service-ca-operator-57cdc4b768-glcdr\" (UID: \"6d3b5566-1d04-4bd1-98eb-e69ace161cd3\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" Sep 17 19:07:14.892054 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.892032 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6d3b5566-1d04-4bd1-98eb-e69ace161cd3-serving-cert\") pod \"service-ca-operator-57cdc4b768-glcdr\" (UID: \"6d3b5566-1d04-4bd1-98eb-e69ace161cd3\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" Sep 17 19:07:14.905293 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.905262 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gtgc\" (UniqueName: \"kubernetes.io/projected/6d3b5566-1d04-4bd1-98eb-e69ace161cd3-kube-api-access-2gtgc\") pod \"service-ca-operator-57cdc4b768-glcdr\" (UID: \"6d3b5566-1d04-4bd1-98eb-e69ace161cd3\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" Sep 17 19:07:14.963300 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.963263 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:14.980496 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:14.980307 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" Sep 17 19:07:15.067231 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.067171 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:15.090976 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.090945 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:15.091419 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:15.091093 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 19:07:15.091419 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:15.091165 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls podName:bce04825-89fe-47fa-a72f-0a24aa333a75 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:19.091146802 +0000 UTC m=+43.044768159 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls") pod "dns-default-5bxxw" (UID: "bce04825-89fe-47fa-a72f-0a24aa333a75") : secret "dns-default-metrics-tls" not found Sep 17 19:07:15.271874 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.271838 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr"] Sep 17 19:07:15.275578 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:07:15.275540 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d3b5566_1d04_4bd1_98eb_e69ace161cd3.slice/crio-a3c345f0225d5fb60d3237ebf212677fd2548ee8c83102ba943443ea57decb9c WatchSource:0}: Error finding container a3c345f0225d5fb60d3237ebf212677fd2548ee8c83102ba943443ea57decb9c: Status 404 returned error can't find the container with id a3c345f0225d5fb60d3237ebf212677fd2548ee8c83102ba943443ea57decb9c Sep 17 19:07:15.594745 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.594661 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:15.594896 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:15.594781 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 19:07:15.594896 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:15.594846 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm podName:5404dd23-6e97-439f-a95d-8b3c8e533de8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:17.594831685 +0000 UTC m=+41.548453040 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hlh59" (UID: "5404dd23-6e97-439f-a95d-8b3c8e533de8") : configmap "kube-rbac-proxy" not found Sep 17 19:07:15.594896 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.594782 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:15.594896 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:15.594869 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 19:07:15.595092 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:15.594954 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls podName:5404dd23-6e97-439f-a95d-8b3c8e533de8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:17.594938126 +0000 UTC m=+41.548559487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hlh59" (UID: "5404dd23-6e97-439f-a95d-8b3c8e533de8") : secret "insights-runtime-extractor-tls" not found Sep 17 19:07:15.685349 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.685313 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-95lc8" event={"ID":"36e00d1e-728d-4377-bca4-5f14d2a88b7e","Type":"ContainerStarted","Data":"61791bc611b87bdbdbe63889960cbd6995e701d9252fb7672d96aaad45693a7c"} Sep 17 19:07:15.686858 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.686827 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" event={"ID":"6d3b5566-1d04-4bd1-98eb-e69ace161cd3","Type":"ContainerStarted","Data":"a3c345f0225d5fb60d3237ebf212677fd2548ee8c83102ba943443ea57decb9c"} Sep 17 19:07:15.687379 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.687334 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:15.709970 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.709885 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-95lc8" podStartSLOduration=2.211999266 podStartE2EDuration="4.709862121s" totalImagesPullingTime="2.497862855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:15.709544641 +0000 UTC m=+39.663166003" watchObservedRunningTime="2026-09-17 19:07:15.709862121 +0000 UTC m=+39.663483482" Sep 17 19:07:15.782263 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.782216 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:15.889665 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.889579 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:15.960022 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.959993 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ndhbz_ac559d16-e42b-4873-bd8e-abda5c4137fd/dns-node-resolver/0.log" Sep 17 19:07:15.988617 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:15.988542 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:16.559865 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:16.559836 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lsmrz_4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a/node-ca/0.log" Sep 17 19:07:16.691353 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:16.691322 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" event={"ID":"33012019-3ef9-475a-a6ab-f7e445f79ea9","Type":"ContainerStarted","Data":"d96223ee61429d33ed4e8532d9d4653b8337dfba31963f4efcab603ac252f4eb"} Sep 17 19:07:17.359047 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:17.359012 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-95lc8_36e00d1e-728d-4377-bca4-5f14d2a88b7e/serve-healthcheck-canary/0.log" Sep 17 19:07:17.610242 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:17.610153 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:17.610242 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:17.610202 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:17.610703 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:17.610266 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 19:07:17.610703 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:17.610312 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 19:07:17.610703 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:17.610318 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm podName:5404dd23-6e97-439f-a95d-8b3c8e533de8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:21.610301078 +0000 UTC m=+45.563922429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hlh59" (UID: "5404dd23-6e97-439f-a95d-8b3c8e533de8") : configmap "kube-rbac-proxy" not found Sep 17 19:07:17.610703 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:17.610450 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls podName:5404dd23-6e97-439f-a95d-8b3c8e533de8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:21.610435329 +0000 UTC m=+45.564056666 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hlh59" (UID: "5404dd23-6e97-439f-a95d-8b3c8e533de8") : secret "insights-runtime-extractor-tls" not found Sep 17 19:07:17.712832 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:17.712783 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" podStartSLOduration=2.383904388 podStartE2EDuration="5.712767776s" totalImagesPullingTime="3.328863388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:17.712182358 +0000 UTC m=+41.665803716" watchObservedRunningTime="2026-09-17 19:07:17.712767776 +0000 UTC m=+41.666389134" Sep 17 19:07:18.698265 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:18.698232 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" event={"ID":"6d3b5566-1d04-4bd1-98eb-e69ace161cd3","Type":"ContainerStarted","Data":"af413186f31fa0a466a09d90b16361838534c52fea2b800193738526c8df2949"} Sep 17 19:07:18.750876 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:18.750822 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" podStartSLOduration=2.264618733 podStartE2EDuration="4.750803593s" totalImagesPullingTime="2.48618486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:18.745210861 +0000 UTC m=+42.698832220" watchObservedRunningTime="2026-09-17 19:07:18.750803593 +0000 UTC m=+42.704424989" Sep 17 19:07:18.864469 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:18.864429 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:18.952202 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:18.952094 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:19.122794 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:19.122760 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:19.122942 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:19.122911 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 19:07:19.123007 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:19.122970 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls podName:bce04825-89fe-47fa-a72f-0a24aa333a75 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:27.122955915 +0000 UTC m=+51.076577252 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls") pod "dns-default-5bxxw" (UID: "bce04825-89fe-47fa-a72f-0a24aa333a75") : secret "dns-default-metrics-tls" not found Sep 17 19:07:19.453127 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:19.453093 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb"] Sep 17 19:07:19.474170 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:19.474137 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb"] Sep 17 19:07:19.474297 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:19.474248 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:19.477847 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:19.477823 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 17 19:07:19.478288 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:19.478274 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ds5js\"" Sep 17 19:07:19.478981 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:19.478957 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 17 19:07:19.626252 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:19.626198 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ddkjb\" (UID: \"9a309853-4490-437d-a9c7-93136e1acbf8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:19.626252 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:19.626256 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9a309853-4490-437d-a9c7-93136e1acbf8-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-ddkjb\" (UID: \"9a309853-4490-437d-a9c7-93136e1acbf8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:19.727241 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:19.727149 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ddkjb\" (UID: \"9a309853-4490-437d-a9c7-93136e1acbf8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:19.727241 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:19.727183 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9a309853-4490-437d-a9c7-93136e1acbf8-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-ddkjb\" (UID: \"9a309853-4490-437d-a9c7-93136e1acbf8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:19.727733 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:19.727300 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 19:07:19.727733 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:19.727365 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert podName:9a309853-4490-437d-a9c7-93136e1acbf8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:20.227348686 +0000 UTC m=+44.180970028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-ddkjb" (UID: "9a309853-4490-437d-a9c7-93136e1acbf8") : secret "networking-console-plugin-cert" not found Sep 17 19:07:19.727855 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:19.727744 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9a309853-4490-437d-a9c7-93136e1acbf8-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-ddkjb\" (UID: \"9a309853-4490-437d-a9c7-93136e1acbf8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:20.002006 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:20.001924 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:20.098145 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:20.098113 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:20.232446 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:20.232415 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ddkjb\" (UID: \"9a309853-4490-437d-a9c7-93136e1acbf8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:20.232552 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:20.232523 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 19:07:20.232611 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:20.232577 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert podName:9a309853-4490-437d-a9c7-93136e1acbf8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:21.232563961 +0000 UTC m=+45.186185298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-ddkjb" (UID: "9a309853-4490-437d-a9c7-93136e1acbf8") : secret "networking-console-plugin-cert" not found Sep 17 19:07:20.453144 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:20.453106 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:20.566935 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:20.566874 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:20.738548 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:20.738456 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:20.847547 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:20.847505 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:21.242270 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:21.242235 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ddkjb\" (UID: \"9a309853-4490-437d-a9c7-93136e1acbf8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:21.242518 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:21.242415 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 19:07:21.242518 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:21.242501 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert podName:9a309853-4490-437d-a9c7-93136e1acbf8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:23.24247877 +0000 UTC m=+47.196100112 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-ddkjb" (UID: "9a309853-4490-437d-a9c7-93136e1acbf8") : secret "networking-console-plugin-cert" not found Sep 17 19:07:21.645104 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:21.645074 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:21.645277 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:21.645115 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:21.645277 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:21.645197 2552 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 19:07:21.645277 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:21.645240 2552 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 19:07:21.645277 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:21.645259 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm podName:5404dd23-6e97-439f-a95d-8b3c8e533de8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:29.64524308 +0000 UTC m=+53.598864437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hlh59" (UID: "5404dd23-6e97-439f-a95d-8b3c8e533de8") : configmap "kube-rbac-proxy" not found Sep 17 19:07:21.645418 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:21.645297 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls podName:5404dd23-6e97-439f-a95d-8b3c8e533de8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:29.645283708 +0000 UTC m=+53.598905045 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hlh59" (UID: "5404dd23-6e97-439f-a95d-8b3c8e533de8") : secret "insights-runtime-extractor-tls" not found Sep 17 19:07:22.092345 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.092313 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:22.305067 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.305031 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-b64lz"] Sep 17 19:07:22.348607 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.348531 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-b64lz"] Sep 17 19:07:22.348751 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.348636 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" Sep 17 19:07:22.354827 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.354808 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 17 19:07:22.355304 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.355290 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 17 19:07:22.355354 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.355323 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 17 19:07:22.355998 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.355985 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-9vz2k\"" Sep 17 19:07:22.356083 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.356069 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 17 19:07:22.381320 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.381291 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:22.451789 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.451755 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b265fbb0-e090-40eb-a53b-72d2c8e5a889-signing-key\") pod \"service-ca-55f6fc7b86-b64lz\" (UID: \"b265fbb0-e090-40eb-a53b-72d2c8e5a889\") " pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" Sep 17 19:07:22.451789 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.451788 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv8dw\" (UniqueName: \"kubernetes.io/projected/b265fbb0-e090-40eb-a53b-72d2c8e5a889-kube-api-access-sv8dw\") pod \"service-ca-55f6fc7b86-b64lz\" (UID: \"b265fbb0-e090-40eb-a53b-72d2c8e5a889\") " pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" Sep 17 19:07:22.451999 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.451850 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b265fbb0-e090-40eb-a53b-72d2c8e5a889-signing-cabundle\") pod \"service-ca-55f6fc7b86-b64lz\" (UID: \"b265fbb0-e090-40eb-a53b-72d2c8e5a889\") " pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" Sep 17 19:07:22.472952 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.472917 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:22.553070 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.553040 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b265fbb0-e090-40eb-a53b-72d2c8e5a889-signing-cabundle\") pod \"service-ca-55f6fc7b86-b64lz\" (UID: \"b265fbb0-e090-40eb-a53b-72d2c8e5a889\") " pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" Sep 17 19:07:22.553243 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.553119 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b265fbb0-e090-40eb-a53b-72d2c8e5a889-signing-key\") pod \"service-ca-55f6fc7b86-b64lz\" (UID: \"b265fbb0-e090-40eb-a53b-72d2c8e5a889\") " pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" Sep 17 19:07:22.553243 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.553137 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv8dw\" (UniqueName: \"kubernetes.io/projected/b265fbb0-e090-40eb-a53b-72d2c8e5a889-kube-api-access-sv8dw\") pod \"service-ca-55f6fc7b86-b64lz\" (UID: \"b265fbb0-e090-40eb-a53b-72d2c8e5a889\") " pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" Sep 17 19:07:22.553696 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.553674 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b265fbb0-e090-40eb-a53b-72d2c8e5a889-signing-cabundle\") pod \"service-ca-55f6fc7b86-b64lz\" (UID: \"b265fbb0-e090-40eb-a53b-72d2c8e5a889\") " pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" Sep 17 19:07:22.555734 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.555715 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b265fbb0-e090-40eb-a53b-72d2c8e5a889-signing-key\") pod \"service-ca-55f6fc7b86-b64lz\" (UID: \"b265fbb0-e090-40eb-a53b-72d2c8e5a889\") " pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" Sep 17 19:07:22.562517 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.562496 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv8dw\" (UniqueName: \"kubernetes.io/projected/b265fbb0-e090-40eb-a53b-72d2c8e5a889-kube-api-access-sv8dw\") pod \"service-ca-55f6fc7b86-b64lz\" (UID: \"b265fbb0-e090-40eb-a53b-72d2c8e5a889\") " pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" Sep 17 19:07:22.657112 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.657026 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" Sep 17 19:07:22.951676 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:22.951634 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-b64lz"] Sep 17 19:07:22.953897 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:07:22.953874 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb265fbb0_e090_40eb_a53b_72d2c8e5a889.slice/crio-4d8f3b76c9e3b00cdbc852d091b676465a6ba583af25873d56c9fe322fde26a5 WatchSource:0}: Error finding container 4d8f3b76c9e3b00cdbc852d091b676465a6ba583af25873d56c9fe322fde26a5: Status 404 returned error can't find the container with id 4d8f3b76c9e3b00cdbc852d091b676465a6ba583af25873d56c9fe322fde26a5 Sep 17 19:07:23.258336 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:23.258253 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ddkjb\" (UID: \"9a309853-4490-437d-a9c7-93136e1acbf8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:23.258672 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:23.258407 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 19:07:23.258672 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:23.258471 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert podName:9a309853-4490-437d-a9c7-93136e1acbf8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:27.258456586 +0000 UTC m=+51.212077927 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-ddkjb" (UID: "9a309853-4490-437d-a9c7-93136e1acbf8") : secret "networking-console-plugin-cert" not found Sep 17 19:07:23.487028 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:23.486993 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:23.711666 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:23.711628 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" event={"ID":"b265fbb0-e090-40eb-a53b-72d2c8e5a889","Type":"ContainerStarted","Data":"5e117b8a013d577ffad6e170819fc555198b3225ab74a35ee00eaa5b20c9f044"} Sep 17 19:07:23.711834 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:23.711692 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" event={"ID":"b265fbb0-e090-40eb-a53b-72d2c8e5a889","Type":"ContainerStarted","Data":"4d8f3b76c9e3b00cdbc852d091b676465a6ba583af25873d56c9fe322fde26a5"} Sep 17 19:07:23.734519 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:23.734472 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-b64lz" podStartSLOduration=1.734457106 podStartE2EDuration="1.734457106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:23.734005265 +0000 UTC m=+47.687626624" watchObservedRunningTime="2026-09-17 19:07:23.734457106 +0000 UTC m=+47.688078466" Sep 17 19:07:24.019444 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:24.019402 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:24.193211 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:24.193176 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:24.371799 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:24.371711 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:24.530106 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:24.530074 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:24.648705 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:24.648624 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:24.752637 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:24.752604 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:25.501999 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:25.501953 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:25.896680 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:25.896641 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:26.305641 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:26.305603 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:26.497205 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:26.497171 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:26.669719 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:26.669628 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:26.739934 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:26.739882 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:26.949121 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:26.949032 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:27.188936 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:27.188891 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:27.189117 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:27.189018 2552 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 19:07:27.189117 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:27.189085 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls podName:bce04825-89fe-47fa-a72f-0a24aa333a75 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:43.189071565 +0000 UTC m=+67.142692901 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls") pod "dns-default-5bxxw" (UID: "bce04825-89fe-47fa-a72f-0a24aa333a75") : secret "dns-default-metrics-tls" not found Sep 17 19:07:27.290080 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:27.290053 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ddkjb\" (UID: \"9a309853-4490-437d-a9c7-93136e1acbf8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:27.290270 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:27.290212 2552 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 19:07:27.290310 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:27.290272 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert podName:9a309853-4490-437d-a9c7-93136e1acbf8 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:35.290257543 +0000 UTC m=+59.243878887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-ddkjb" (UID: "9a309853-4490-437d-a9c7-93136e1acbf8") : secret "networking-console-plugin-cert" not found Sep 17 19:07:27.592396 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:27.592308 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:27.692319 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:27.692288 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:27.791848 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:27.791812 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:27.917387 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:27.917300 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:28.287552 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:28.287524 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:28.508162 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:28.508123 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:28.579121 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:28.579037 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:29.085589 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:29.085554 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:29.490284 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:29.490199 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:29.707930 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:29.707855 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:29.707930 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:29.707949 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:29.708527 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:29.708504 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5404dd23-6e97-439f-a95d-8b3c8e533de8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:29.710393 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:29.710370 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5404dd23-6e97-439f-a95d-8b3c8e533de8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hlh59\" (UID: \"5404dd23-6e97-439f-a95d-8b3c8e533de8\") " pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:29.762934 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:29.762885 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hlh59" Sep 17 19:07:29.867787 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:29.867745 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:29.938651 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:29.938613 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:30.043616 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:30.043589 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hlh59"] Sep 17 19:07:30.313502 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:30.313408 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:30.526002 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:30.525926 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:30.733940 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:30.733912 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hlh59" event={"ID":"5404dd23-6e97-439f-a95d-8b3c8e533de8","Type":"ContainerStarted","Data":"b8f95b46cd3d199ddafb156aaf2b1b5c7c5da0f9bd958f4dc370cb3cd3424e4b"} Sep 17 19:07:30.734093 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:30.733971 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hlh59" event={"ID":"5404dd23-6e97-439f-a95d-8b3c8e533de8","Type":"ContainerStarted","Data":"438b9d46b8ae53398a5b8cb323ce369d34fb80e877099e7a9a46372ac03524a6"} Sep 17 19:07:30.734093 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:30.733990 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hlh59" event={"ID":"5404dd23-6e97-439f-a95d-8b3c8e533de8","Type":"ContainerStarted","Data":"4f0a75fd6c092c1ea9bfbc89bf07b8aa2e46b24c795254df74b548fa7c8947e2"} Sep 17 19:07:31.093510 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:31.093477 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:31.485577 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:31.485477 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:31.701765 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:31.701681 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:32.093481 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:32.093446 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:32.495727 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:32.495697 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:32.742303 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:32.742270 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hlh59" event={"ID":"5404dd23-6e97-439f-a95d-8b3c8e533de8","Type":"ContainerStarted","Data":"fd11aa3dfdd4ac5e4572e87ee827a63cf464263f0415fdc6653178d2afd9b6cf"} Sep 17 19:07:32.762522 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:32.762482 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hlh59" podStartSLOduration=17.993930253 podStartE2EDuration="19.762466998s" totalImagesPullingTime="1.768536745s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:07:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:32.761154783 +0000 UTC m=+56.714776142" watchObservedRunningTime="2026-09-17 19:07:32.762466998 +0000 UTC m=+56.716088357" Sep 17 19:07:33.103124 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:33.103030 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:33.405559 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:33.405470 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:33.526981 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:33.526952 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:33.654222 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:33.654194 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:33.754430 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:33.754335 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:33.842566 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:33.842540 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:33.970944 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:33.970917 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:34.099603 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:34.099570 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:34.295603 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:34.295572 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:34.385241 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:34.385158 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:34.524109 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:34.524066 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:34.718535 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:34.718454 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:34.820458 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:34.820415 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:34.917574 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:34.917525 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:35.182657 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:35.182620 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:35.353942 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:35.353892 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ddkjb\" (UID: \"9a309853-4490-437d-a9c7-93136e1acbf8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:35.356599 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:35.356570 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9a309853-4490-437d-a9c7-93136e1acbf8-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ddkjb\" (UID: \"9a309853-4490-437d-a9c7-93136e1acbf8\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:35.382521 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:35.382491 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" Sep 17 19:07:35.391539 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:35.391504 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:35.495288 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:35.495248 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:35.678143 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:35.678114 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb"] Sep 17 19:07:35.681094 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:07:35.681066 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a309853_4490_437d_a9c7_93136e1acbf8.slice/crio-b6a155d0535a495cc9da19b813854f3e2434de6d4005d551acdf72bd03396bf9 WatchSource:0}: Error finding container b6a155d0535a495cc9da19b813854f3e2434de6d4005d551acdf72bd03396bf9: Status 404 returned error can't find the container with id b6a155d0535a495cc9da19b813854f3e2434de6d4005d551acdf72bd03396bf9 Sep 17 19:07:35.752991 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:35.752895 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" event={"ID":"9a309853-4490-437d-a9c7-93136e1acbf8","Type":"ContainerStarted","Data":"b6a155d0535a495cc9da19b813854f3e2434de6d4005d551acdf72bd03396bf9"} Sep 17 19:07:36.654781 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:36.654747 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jt5kn" Sep 17 19:07:37.761110 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:37.761076 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" event={"ID":"9a309853-4490-437d-a9c7-93136e1acbf8","Type":"ContainerStarted","Data":"cdf43eed594f2aaf8d077e5350646c21ec157a902efd86bbba4508a13b5f4bb3"} Sep 17 19:07:37.778375 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:37.778319 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-ddkjb" podStartSLOduration=17.704460231 podStartE2EDuration="18.778297128s" totalImagesPullingTime="1.073836897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:37.777219352 +0000 UTC m=+61.730840711" watchObservedRunningTime="2026-09-17 19:07:37.778297128 +0000 UTC m=+61.731918489" Sep 17 19:07:41.675147 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:41.675116 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:41.747440 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:41.747405 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:42.202544 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.202498 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v48sq\" (UniqueName: \"kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq\") pod \"network-check-target-t7jj9\" (UID: \"45c9424d-9eba-4232-9e83-85dff9e0132f\") " pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:42.205656 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.205634 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 19:07:42.216069 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.216043 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 19:07:42.226077 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.226050 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v48sq\" (UniqueName: \"kubernetes.io/projected/45c9424d-9eba-4232-9e83-85dff9e0132f-kube-api-access-v48sq\") pod \"network-check-target-t7jj9\" (UID: \"45c9424d-9eba-4232-9e83-85dff9e0132f\") " pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:42.234043 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.234027 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-b9ftw\"" Sep 17 19:07:42.242287 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.242272 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:42.303212 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.303179 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:07:42.303366 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.303258 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:07:42.307576 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.307249 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 19:07:42.307685 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.307579 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 19:07:42.316572 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.316516 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c5ac3838-c267-4741-8925-8c2e90745194-original-pull-secret\") pod \"global-pull-secret-syncer-ls7s4\" (UID: \"c5ac3838-c267-4741-8925-8c2e90745194\") " pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:07:42.316883 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.316839 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/366e0b69-56de-45e9-9a8b-87f0a4c8b4bf-metrics-certs\") pod \"network-metrics-daemon-422mp\" (UID: \"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf\") " pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:07:42.522689 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.522663 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-47zq2\"" Sep 17 19:07:42.525580 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.525565 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ls7s4" Sep 17 19:07:42.530232 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.530215 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-422mp" Sep 17 19:07:42.544335 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.544314 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-t7jj9"] Sep 17 19:07:42.547336 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:07:42.547316 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45c9424d_9eba_4232_9e83_85dff9e0132f.slice/crio-b75c9feb8eebf040cc412d8f2ce710f4e9b9d9ce8d6f8206236917c13fa14dfd WatchSource:0}: Error finding container b75c9feb8eebf040cc412d8f2ce710f4e9b9d9ce8d6f8206236917c13fa14dfd: Status 404 returned error can't find the container with id b75c9feb8eebf040cc412d8f2ce710f4e9b9d9ce8d6f8206236917c13fa14dfd Sep 17 19:07:42.777502 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.777417 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t7jj9" event={"ID":"45c9424d-9eba-4232-9e83-85dff9e0132f","Type":"ContainerStarted","Data":"b75c9feb8eebf040cc412d8f2ce710f4e9b9d9ce8d6f8206236917c13fa14dfd"} Sep 17 19:07:42.795095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.795061 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:42.819921 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.819873 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ls7s4"] Sep 17 19:07:42.822610 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:07:42.822584 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5ac3838_c267_4741_8925_8c2e90745194.slice/crio-acdb966caf1695493cf58a2c28e8504ec651cfd6828a8c72564694da136fa812 WatchSource:0}: Error finding container acdb966caf1695493cf58a2c28e8504ec651cfd6828a8c72564694da136fa812: Status 404 returned error can't find the container with id acdb966caf1695493cf58a2c28e8504ec651cfd6828a8c72564694da136fa812 Sep 17 19:07:42.856704 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:42.856549 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-422mp"] Sep 17 19:07:42.858882 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:07:42.858858 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod366e0b69_56de_45e9_9a8b_87f0a4c8b4bf.slice/crio-44d130e292290b11cbcf2475acc445f3f1179b8766d211b1dde7bd9a0aceb430 WatchSource:0}: Error finding container 44d130e292290b11cbcf2475acc445f3f1179b8766d211b1dde7bd9a0aceb430: Status 404 returned error can't find the container with id 44d130e292290b11cbcf2475acc445f3f1179b8766d211b1dde7bd9a0aceb430 Sep 17 19:07:43.210526 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:43.210435 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:43.213149 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:43.213123 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bce04825-89fe-47fa-a72f-0a24aa333a75-metrics-tls\") pod \"dns-default-5bxxw\" (UID: \"bce04825-89fe-47fa-a72f-0a24aa333a75\") " pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:43.494540 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:43.494405 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dpmzt\"" Sep 17 19:07:43.502047 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:43.502024 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:43.786301 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:43.786261 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-422mp" event={"ID":"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf","Type":"ContainerStarted","Data":"44d130e292290b11cbcf2475acc445f3f1179b8766d211b1dde7bd9a0aceb430"} Sep 17 19:07:43.788385 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:43.788358 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ls7s4" event={"ID":"c5ac3838-c267-4741-8925-8c2e90745194","Type":"ContainerStarted","Data":"acdb966caf1695493cf58a2c28e8504ec651cfd6828a8c72564694da136fa812"} Sep 17 19:07:43.808507 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:43.808384 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5bxxw"] Sep 17 19:07:44.149979 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:07:44.146036 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbce04825_89fe_47fa_a72f_0a24aa333a75.slice/crio-10fca94f0ea7a6b7db71c09819a4590e322a4377120b2f8cfb2e93e675a09af6 WatchSource:0}: Error finding container 10fca94f0ea7a6b7db71c09819a4590e322a4377120b2f8cfb2e93e675a09af6: Status 404 returned error can't find the container with id 10fca94f0ea7a6b7db71c09819a4590e322a4377120b2f8cfb2e93e675a09af6 Sep 17 19:07:44.593443 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:44.593414 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:44.694975 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:44.694945 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:44.796704 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:44.796669 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-422mp" event={"ID":"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf","Type":"ContainerStarted","Data":"352a130a247069f5da9b1135833625590fbe283486b3bc3d80b3063aa075f17e"} Sep 17 19:07:44.798507 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:44.798449 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5bxxw" event={"ID":"bce04825-89fe-47fa-a72f-0a24aa333a75","Type":"ContainerStarted","Data":"10fca94f0ea7a6b7db71c09819a4590e322a4377120b2f8cfb2e93e675a09af6"} Sep 17 19:07:45.563712 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:45.563675 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:45.678352 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:45.678294 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:45.765502 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:45.765470 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:45.804773 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:45.804735 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-422mp" event={"ID":"366e0b69-56de-45e9-9a8b-87f0a4c8b4bf","Type":"ContainerStarted","Data":"02d995fbbb448337529ac06cc55e96a8e9c0a077e692339b5eaba935305e259b"} Sep 17 19:07:45.824481 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:45.824289 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-422mp" podStartSLOduration=67.202346721 podStartE2EDuration="1m8.824267729s" totalImagesPullingTime="1.621921008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:45.823853997 +0000 UTC m=+69.777475368" watchObservedRunningTime="2026-09-17 19:07:45.824267729 +0000 UTC m=+69.777889086" Sep 17 19:07:45.865945 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:45.865897 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:46.004949 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.004883 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:46.276033 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.275982 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh"] Sep 17 19:07:46.278211 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.278190 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh" Sep 17 19:07:46.281948 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.281923 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-l6dms\"" Sep 17 19:07:46.281948 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.281935 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 17 19:07:46.282932 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.282882 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 17 19:07:46.283056 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.283022 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 17 19:07:46.283870 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.283719 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 17 19:07:46.290641 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.290612 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh"] Sep 17 19:07:46.300717 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.300693 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl"] Sep 17 19:07:46.307572 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.307548 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.310562 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.310541 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 17 19:07:46.310691 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.310566 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 17 19:07:46.310893 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.310870 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 17 19:07:46.311102 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.311085 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 17 19:07:46.320003 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.319456 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl"] Sep 17 19:07:46.345064 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.342937 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78d97bc468-r4kqh\" (UID: \"47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh" Sep 17 19:07:46.345379 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.345357 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsmv2\" (UniqueName: \"kubernetes.io/projected/47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4-kube-api-access-tsmv2\") pod \"managed-serviceaccount-addon-agent-78d97bc468-r4kqh\" (UID: \"47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh" Sep 17 19:07:46.446757 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.446725 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78d97bc468-r4kqh\" (UID: \"47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh" Sep 17 19:07:46.446757 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.446763 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsmv2\" (UniqueName: \"kubernetes.io/projected/47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4-kube-api-access-tsmv2\") pod \"managed-serviceaccount-addon-agent-78d97bc468-r4kqh\" (UID: \"47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh" Sep 17 19:07:46.447070 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.446808 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnr8c\" (UniqueName: \"kubernetes.io/projected/e2a0c892-588a-425f-9d0f-a249b3304326-kube-api-access-nnr8c\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.447070 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.446836 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e2a0c892-588a-425f-9d0f-a249b3304326-hub\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.447070 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.446860 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2a0c892-588a-425f-9d0f-a249b3304326-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.447070 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.446997 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2a0c892-588a-425f-9d0f-a249b3304326-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.447070 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.447036 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e2a0c892-588a-425f-9d0f-a249b3304326-ca\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.447070 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.447065 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e2a0c892-588a-425f-9d0f-a249b3304326-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.449747 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.449668 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-78d97bc468-r4kqh\" (UID: \"47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh" Sep 17 19:07:46.469293 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.469264 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsmv2\" (UniqueName: \"kubernetes.io/projected/47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4-kube-api-access-tsmv2\") pod \"managed-serviceaccount-addon-agent-78d97bc468-r4kqh\" (UID: \"47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh" Sep 17 19:07:46.513762 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.513727 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:46.548443 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.548365 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2a0c892-588a-425f-9d0f-a249b3304326-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.548443 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.548407 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e2a0c892-588a-425f-9d0f-a249b3304326-ca\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.548443 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.548429 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e2a0c892-588a-425f-9d0f-a249b3304326-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.548702 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.548484 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnr8c\" (UniqueName: \"kubernetes.io/projected/e2a0c892-588a-425f-9d0f-a249b3304326-kube-api-access-nnr8c\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.548702 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.548509 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e2a0c892-588a-425f-9d0f-a249b3304326-hub\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.548702 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.548535 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2a0c892-588a-425f-9d0f-a249b3304326-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.549244 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.549194 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2a0c892-588a-425f-9d0f-a249b3304326-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.551716 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.551643 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e2a0c892-588a-425f-9d0f-a249b3304326-ca\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.552356 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.552331 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2a0c892-588a-425f-9d0f-a249b3304326-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.552480 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.552418 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e2a0c892-588a-425f-9d0f-a249b3304326-hub\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.552480 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.552451 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e2a0c892-588a-425f-9d0f-a249b3304326-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.561031 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.561010 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnr8c\" (UniqueName: \"kubernetes.io/projected/e2a0c892-588a-425f-9d0f-a249b3304326-kube-api-access-nnr8c\") pod \"cluster-proxy-proxy-agent-54f9f5d568-42gkl\" (UID: \"e2a0c892-588a-425f-9d0f-a249b3304326\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.601514 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.601485 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh" Sep 17 19:07:46.620487 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.620456 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" Sep 17 19:07:46.627689 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.627661 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:46.761162 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.761121 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:46.864594 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:46.864507 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:47.087803 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:47.087715 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:47.186307 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:47.186226 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:47.594513 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:47.594473 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:47.989137 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:47.989055 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:48.090878 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.090849 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:48.166184 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.166095 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl"] Sep 17 19:07:48.169287 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.168510 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh"] Sep 17 19:07:48.169599 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:07:48.169576 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2a0c892_588a_425f_9d0f_a249b3304326.slice/crio-14bba9ef8f46d3158426ccd1b3b773ce4dd441c8efdf78b652b533bcd92a27ce WatchSource:0}: Error finding container 14bba9ef8f46d3158426ccd1b3b773ce4dd441c8efdf78b652b533bcd92a27ce: Status 404 returned error can't find the container with id 14bba9ef8f46d3158426ccd1b3b773ce4dd441c8efdf78b652b533bcd92a27ce Sep 17 19:07:48.173272 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:07:48.173253 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47bd3be2_30d9_455e_9a54_cdd0e5b6b1b4.slice/crio-e94b1133b93aeb7b49d65e7bf63b3745fa0b70a28824faeb4aecc53bde412332 WatchSource:0}: Error finding container e94b1133b93aeb7b49d65e7bf63b3745fa0b70a28824faeb4aecc53bde412332: Status 404 returned error can't find the container with id e94b1133b93aeb7b49d65e7bf63b3745fa0b70a28824faeb4aecc53bde412332 Sep 17 19:07:48.819040 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.818224 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-t7jj9" event={"ID":"45c9424d-9eba-4232-9e83-85dff9e0132f","Type":"ContainerStarted","Data":"0ae634fb16103cebf34a376b9bc287493895334ac23015a3015c5906c467f623"} Sep 17 19:07:48.819040 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.818988 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:07:48.821734 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.821681 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" event={"ID":"e2a0c892-588a-425f-9d0f-a249b3304326","Type":"ContainerStarted","Data":"14bba9ef8f46d3158426ccd1b3b773ce4dd441c8efdf78b652b533bcd92a27ce"} Sep 17 19:07:48.824200 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.824177 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh" event={"ID":"47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4","Type":"ContainerStarted","Data":"e94b1133b93aeb7b49d65e7bf63b3745fa0b70a28824faeb4aecc53bde412332"} Sep 17 19:07:48.826721 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.826695 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ls7s4" event={"ID":"c5ac3838-c267-4741-8925-8c2e90745194","Type":"ContainerStarted","Data":"a53d89296840b80f050bd984eb36052b2fe8caf825f2a9fbde9b428353571035"} Sep 17 19:07:48.832298 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.832270 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5bxxw" event={"ID":"bce04825-89fe-47fa-a72f-0a24aa333a75","Type":"ContainerStarted","Data":"19da8ade39ff750943e32a3a126f143d18e89fcba03ff14cce30fb2a02d72d1e"} Sep 17 19:07:48.832408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.832324 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5bxxw" event={"ID":"bce04825-89fe-47fa-a72f-0a24aa333a75","Type":"ContainerStarted","Data":"c2a356e4cc0a71274bb7ad54107e4b56e2f3225302bc38ecf491953a2190ec0a"} Sep 17 19:07:48.832880 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.832860 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:48.866486 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.866414 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-t7jj9" podStartSLOduration=66.536163178 podStartE2EDuration="1m11.866391793s" totalImagesPullingTime="5.330228615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:48.839371344 +0000 UTC m=+72.792992705" watchObservedRunningTime="2026-09-17 19:07:48.866391793 +0000 UTC m=+72.820013153" Sep 17 19:07:48.866924 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.866668 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ls7s4" podStartSLOduration=66.800437588 podStartE2EDuration="1m11.866659157s" totalImagesPullingTime="5.066221569s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:48.86514621 +0000 UTC m=+72.818767570" watchObservedRunningTime="2026-09-17 19:07:48.866659157 +0000 UTC m=+72.820280517" Sep 17 19:07:48.889178 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:48.889119 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5bxxw" podStartSLOduration=34.159298949 podStartE2EDuration="37.889102956s" totalImagesPullingTime="3.729804007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:48.88597049 +0000 UTC m=+72.839591847" watchObservedRunningTime="2026-09-17 19:07:48.889102956 +0000 UTC m=+72.842724314" Sep 17 19:07:49.224498 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:49.224423 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:49.320516 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:49.320452 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:50.815035 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:50.815000 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:50.896307 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:50.896258 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:50.981156 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:50.981126 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:52.850240 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:52.850205 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" event={"ID":"e2a0c892-588a-425f-9d0f-a249b3304326","Type":"ContainerStarted","Data":"50b58c312ea9305ba9fb8cc4dc43de7480e68c4f178d68136dedc3decec29d5b"} Sep 17 19:07:52.850672 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:52.850253 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" event={"ID":"e2a0c892-588a-425f-9d0f-a249b3304326","Type":"ContainerStarted","Data":"d82092e9233daab10ddcb72efa78aa0614b1e952bcc50f633da3dd9a8e9f5e73"} Sep 17 19:07:52.850672 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:52.850274 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" event={"ID":"e2a0c892-588a-425f-9d0f-a249b3304326","Type":"ContainerStarted","Data":"212b6e445fbe8969c7528f57aeb10a3520d2e1e8e96c6a5554af1e54c83cbf50"} Sep 17 19:07:52.851768 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:52.851745 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh" event={"ID":"47bd3be2-30d9-455e-9a54-cdd0e5b6b1b4","Type":"ContainerStarted","Data":"42fe2c0afaea3a721dfa8ee9d7ef3010716e7681ae5a950db8d8511cd6787bfb"} Sep 17 19:07:52.898396 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:52.898331 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54f9f5d568-42gkl" podStartSLOduration=2.993467188 podStartE2EDuration="6.898310292s" totalImagesPullingTime="3.904843104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:52.897003628 +0000 UTC m=+76.850624987" watchObservedRunningTime="2026-09-17 19:07:52.898310292 +0000 UTC m=+76.851931652" Sep 17 19:07:52.924065 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:52.924016 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-78d97bc468-r4kqh" podStartSLOduration=3.032592973 podStartE2EDuration="6.923999312s" totalImagesPullingTime="3.891406339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:52.922116121 +0000 UTC m=+76.875737480" watchObservedRunningTime="2026-09-17 19:07:52.923999312 +0000 UTC m=+76.877620670" Sep 17 19:07:52.936863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:52.936836 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:54.205362 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:54.205316 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:54.272919 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:54.272879 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:54.980072 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:54.980034 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:55.142113 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.142076 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:55.304008 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.303968 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:55.367573 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.367541 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mjckh"] Sep 17 19:07:55.370934 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.370917 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.377225 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.377203 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 17 19:07:55.377967 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.377952 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 17 19:07:55.378205 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.378191 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 17 19:07:55.378573 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.378545 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 17 19:07:55.379208 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.379192 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 17 19:07:55.380017 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.380000 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2zq9r\"" Sep 17 19:07:55.380125 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.380020 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 17 19:07:55.425246 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.425216 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:55.520302 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.520265 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-tls\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.520302 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.520305 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n97dt\" (UniqueName: \"kubernetes.io/projected/568754ba-19bf-4224-b0b0-2bfbfbee6011-kube-api-access-n97dt\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.520524 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.520327 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-wtmp\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.520524 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.520352 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/568754ba-19bf-4224-b0b0-2bfbfbee6011-root\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.520524 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.520418 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-textfile\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.520524 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.520439 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/568754ba-19bf-4224-b0b0-2bfbfbee6011-sys\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.520524 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.520455 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.520524 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.520482 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-accelerators-collector-config\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.520737 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.520576 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/568754ba-19bf-4224-b0b0-2bfbfbee6011-metrics-client-ca\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.524226 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.524199 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:55.621805 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.621717 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/568754ba-19bf-4224-b0b0-2bfbfbee6011-sys\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.621805 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.621753 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.621805 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.621783 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-accelerators-collector-config\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.622087 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.621821 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/568754ba-19bf-4224-b0b0-2bfbfbee6011-metrics-client-ca\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.622087 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.621828 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/568754ba-19bf-4224-b0b0-2bfbfbee6011-sys\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.622087 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.621848 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-tls\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.622087 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.621888 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n97dt\" (UniqueName: \"kubernetes.io/projected/568754ba-19bf-4224-b0b0-2bfbfbee6011-kube-api-access-n97dt\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.622087 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.621939 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-wtmp\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.622087 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:55.621954 2552 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 17 19:07:55.622087 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.621964 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/568754ba-19bf-4224-b0b0-2bfbfbee6011-root\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.622087 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.622000 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-textfile\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.622087 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:07:55.622029 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-tls podName:568754ba-19bf-4224-b0b0-2bfbfbee6011 nodeName:}" failed. No retries permitted until 2026-09-17 19:07:56.122002276 +0000 UTC m=+80.075623628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-tls") pod "node-exporter-mjckh" (UID: "568754ba-19bf-4224-b0b0-2bfbfbee6011") : secret "node-exporter-tls" not found Sep 17 19:07:55.622511 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.622327 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-wtmp\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.622561 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.622522 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-accelerators-collector-config\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.622620 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.622557 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/568754ba-19bf-4224-b0b0-2bfbfbee6011-metrics-client-ca\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.622684 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.622621 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/568754ba-19bf-4224-b0b0-2bfbfbee6011-root\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.622744 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.622711 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-textfile\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.624730 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.624707 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.631273 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.631251 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n97dt\" (UniqueName: \"kubernetes.io/projected/568754ba-19bf-4224-b0b0-2bfbfbee6011-kube-api-access-n97dt\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:55.635596 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.635500 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:55.750266 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.750230 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:55.848093 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.848063 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:55.924435 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:55.924335 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:56.125545 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:56.125512 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-tls\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:56.127921 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:56.127868 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/568754ba-19bf-4224-b0b0-2bfbfbee6011-node-exporter-tls\") pod \"node-exporter-mjckh\" (UID: \"568754ba-19bf-4224-b0b0-2bfbfbee6011\") " pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:56.208553 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:56.208471 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:56.282145 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:56.282112 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mjckh" Sep 17 19:07:56.294420 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:07:56.294387 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568754ba_19bf_4224_b0b0_2bfbfbee6011.slice/crio-9234be541958aa4138fadae30b468c0a6af2dfd4a1fe3d9509014695d32f21aa WatchSource:0}: Error finding container 9234be541958aa4138fadae30b468c0a6af2dfd4a1fe3d9509014695d32f21aa: Status 404 returned error can't find the container with id 9234be541958aa4138fadae30b468c0a6af2dfd4a1fe3d9509014695d32f21aa Sep 17 19:07:56.297026 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:56.297008 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:56.428555 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:56.428517 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:56.866643 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:56.866601 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mjckh" event={"ID":"568754ba-19bf-4224-b0b0-2bfbfbee6011","Type":"ContainerStarted","Data":"9234be541958aa4138fadae30b468c0a6af2dfd4a1fe3d9509014695d32f21aa"} Sep 17 19:07:57.796882 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:57.796842 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:57.871492 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:57.871377 2552 generic.go:356] "Generic (PLEG): container finished" podID="568754ba-19bf-4224-b0b0-2bfbfbee6011" containerID="a5a8d36e0fc6640c794585164e0c0ad61b367bbb11c248f2d54514a9b2074d28" exitCode=0 Sep 17 19:07:57.871492 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:57.871420 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mjckh" event={"ID":"568754ba-19bf-4224-b0b0-2bfbfbee6011","Type":"ContainerDied","Data":"a5a8d36e0fc6640c794585164e0c0ad61b367bbb11c248f2d54514a9b2074d28"} Sep 17 19:07:57.897491 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:57.897465 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:58.232378 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:58.232291 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:58.332290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:58.332256 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:58.434872 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:58.434838 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:58.876611 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:58.876579 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mjckh" event={"ID":"568754ba-19bf-4224-b0b0-2bfbfbee6011","Type":"ContainerStarted","Data":"954d0e6e34987705622778677b9b19649f55dee00646d3cff71f993386f17cde"} Sep 17 19:07:58.877103 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:58.876628 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mjckh" event={"ID":"568754ba-19bf-4224-b0b0-2bfbfbee6011","Type":"ContainerStarted","Data":"7bfb99915e442d5c0ae3d8b0d2d2157f6e03b2223bd46e17708997069121b29c"} Sep 17 19:07:58.897691 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:58.897662 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:59.790363 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:59.790334 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:07:59.843759 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:59.843729 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5bxxw" Sep 17 19:07:59.870770 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:59.870723 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mjckh" podStartSLOduration=3.946682243 podStartE2EDuration="4.870704357s" totalImagesPullingTime="855.697481ms" totalInitContainerRuntime="68.324633ms" isStatefulPod=true podCreationTimestamp="2026-09-17 19:07:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:07:58.896667806 +0000 UTC m=+82.850289165" watchObservedRunningTime="2026-09-17 19:07:59.870704357 +0000 UTC m=+83.824325758" Sep 17 19:07:59.897283 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:07:59.897250 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:00.022765 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.022734 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:00.060065 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.059985 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-9bh24"] Sep 17 19:08:00.063068 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.063049 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" Sep 17 19:08:00.066560 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.066542 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-cfzk8\"" Sep 17 19:08:00.066679 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.066624 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 17 19:08:00.079115 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.079088 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-9bh24"] Sep 17 19:08:00.120618 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.120595 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:00.156181 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.156147 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96190262-af6d-4736-a306-9f1dc4014513-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-9bh24\" (UID: \"96190262-af6d-4736-a306-9f1dc4014513\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" Sep 17 19:08:00.156337 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.156220 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nrkq\" (UniqueName: \"kubernetes.io/projected/96190262-af6d-4736-a306-9f1dc4014513-kube-api-access-4nrkq\") pod \"monitoring-plugin-5984459ff8-9bh24\" (UID: \"96190262-af6d-4736-a306-9f1dc4014513\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" Sep 17 19:08:00.261539 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.257710 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96190262-af6d-4736-a306-9f1dc4014513-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-9bh24\" (UID: \"96190262-af6d-4736-a306-9f1dc4014513\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" Sep 17 19:08:00.261539 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.257863 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nrkq\" (UniqueName: \"kubernetes.io/projected/96190262-af6d-4736-a306-9f1dc4014513-kube-api-access-4nrkq\") pod \"monitoring-plugin-5984459ff8-9bh24\" (UID: \"96190262-af6d-4736-a306-9f1dc4014513\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" Sep 17 19:08:00.261539 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:08:00.257937 2552 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 17 19:08:00.261539 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:08:00.258015 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96190262-af6d-4736-a306-9f1dc4014513-monitoring-plugin-cert podName:96190262-af6d-4736-a306-9f1dc4014513 nodeName:}" failed. No retries permitted until 2026-09-17 19:08:00.757992631 +0000 UTC m=+84.711613982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/96190262-af6d-4736-a306-9f1dc4014513-monitoring-plugin-cert") pod "monitoring-plugin-5984459ff8-9bh24" (UID: "96190262-af6d-4736-a306-9f1dc4014513") : secret "monitoring-plugin-cert" not found Sep 17 19:08:00.269544 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.269519 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nrkq\" (UniqueName: \"kubernetes.io/projected/96190262-af6d-4736-a306-9f1dc4014513-kube-api-access-4nrkq\") pod \"monitoring-plugin-5984459ff8-9bh24\" (UID: \"96190262-af6d-4736-a306-9f1dc4014513\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" Sep 17 19:08:00.587720 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.587684 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:00.760912 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.760867 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96190262-af6d-4736-a306-9f1dc4014513-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-9bh24\" (UID: \"96190262-af6d-4736-a306-9f1dc4014513\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" Sep 17 19:08:00.763518 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.763488 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/96190262-af6d-4736-a306-9f1dc4014513-monitoring-plugin-cert\") pod \"monitoring-plugin-5984459ff8-9bh24\" (UID: \"96190262-af6d-4736-a306-9f1dc4014513\") " pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" Sep 17 19:08:00.820553 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.820521 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:00.922414 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.922325 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:00.972312 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:00.972285 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" Sep 17 19:08:01.239610 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:01.239519 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5984459ff8-9bh24"] Sep 17 19:08:01.242082 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:08:01.242052 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96190262_af6d_4736_a306_9f1dc4014513.slice/crio-e742b30d22e0af74c0568059c03a2526a77d222693ca541544c57fa78892e40b WatchSource:0}: Error finding container e742b30d22e0af74c0568059c03a2526a77d222693ca541544c57fa78892e40b: Status 404 returned error can't find the container with id e742b30d22e0af74c0568059c03a2526a77d222693ca541544c57fa78892e40b Sep 17 19:08:01.583807 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:01.583772 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:01.811340 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:01.811304 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:01.889219 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:01.889145 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" event={"ID":"96190262-af6d-4736-a306-9f1dc4014513","Type":"ContainerStarted","Data":"e742b30d22e0af74c0568059c03a2526a77d222693ca541544c57fa78892e40b"} Sep 17 19:08:01.910517 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:01.910482 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:02.894893 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:02.894813 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" event={"ID":"96190262-af6d-4736-a306-9f1dc4014513","Type":"ContainerStarted","Data":"29d6be0012f127e0e6f72a00e20ce3bd491d1afc71afb85fa1bbdba7dbad914d"} Sep 17 19:08:02.895279 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:02.895019 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" Sep 17 19:08:02.899456 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:02.899439 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" Sep 17 19:08:02.912119 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:02.912079 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5984459ff8-9bh24" podStartSLOduration=1.54324905 podStartE2EDuration="2.912064849s" totalImagesPullingTime="1.368815799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:08:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:08:02.911661818 +0000 UTC m=+86.865283176" watchObservedRunningTime="2026-09-17 19:08:02.912064849 +0000 UTC m=+86.865686208" Sep 17 19:08:03.805053 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:03.805020 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:03.882147 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:03.882118 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:05.797410 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:05.797378 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:05.897437 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:05.897405 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:07.604804 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:07.604754 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:07.663297 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:07.663266 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:08.636282 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:08.636248 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:08.729408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:08.729375 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:09.755639 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:09.755610 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:09.860763 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:09.860719 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:10.040103 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:10.040068 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:10.146639 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:10.146602 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:11.004290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:11.004258 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:11.104108 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:11.104066 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:11.662286 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:11.662252 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:11.765147 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:11.765116 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:14.137241 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:14.137206 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:14.238159 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:14.238123 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:17.700746 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:17.700710 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:17.798911 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:17.798876 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:18.570399 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:18.570360 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:18.670116 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:18.670079 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:18.731225 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:18.731195 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:20.844729 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:20.844702 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-t7jj9" Sep 17 19:08:21.215659 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:21.215564 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:21.315961 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:21.315897 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:21.416506 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:21.416465 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:26.659090 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:26.659041 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:26.717690 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:26.717650 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:28.980883 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:28.980852 2552 generic.go:356] "Generic (PLEG): container finished" podID="33012019-3ef9-475a-a6ab-f7e445f79ea9" containerID="d96223ee61429d33ed4e8532d9d4653b8337dfba31963f4efcab603ac252f4eb" exitCode=0 Sep 17 19:08:28.981299 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:28.980892 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" event={"ID":"33012019-3ef9-475a-a6ab-f7e445f79ea9","Type":"ContainerDied","Data":"d96223ee61429d33ed4e8532d9d4653b8337dfba31963f4efcab603ac252f4eb"} Sep 17 19:08:28.981299 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:28.981196 2552 scope.go:117] "RemoveContainer" containerID="d96223ee61429d33ed4e8532d9d4653b8337dfba31963f4efcab603ac252f4eb" Sep 17 19:08:29.986096 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:29.986050 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-zsjnd" event={"ID":"33012019-3ef9-475a-a6ab-f7e445f79ea9","Type":"ContainerStarted","Data":"29f04efc859a423d7649de5651043c2580b66ffa468c71cea0f718b67efafdc0"} Sep 17 19:08:34.000556 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:34.000522 2552 generic.go:356] "Generic (PLEG): container finished" podID="6d3b5566-1d04-4bd1-98eb-e69ace161cd3" containerID="af413186f31fa0a466a09d90b16361838534c52fea2b800193738526c8df2949" exitCode=0 Sep 17 19:08:34.000888 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:34.000594 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" event={"ID":"6d3b5566-1d04-4bd1-98eb-e69ace161cd3","Type":"ContainerDied","Data":"af413186f31fa0a466a09d90b16361838534c52fea2b800193738526c8df2949"} Sep 17 19:08:34.000888 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:34.000877 2552 scope.go:117] "RemoveContainer" containerID="af413186f31fa0a466a09d90b16361838534c52fea2b800193738526c8df2949" Sep 17 19:08:35.009488 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:35.009452 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-glcdr" event={"ID":"6d3b5566-1d04-4bd1-98eb-e69ace161cd3","Type":"ContainerStarted","Data":"45f61b301286f2bc9c10eaed4cd9374a4dcff71022109c5c54d7a2e291ecb02a"} Sep 17 19:08:36.215968 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:36.215929 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:08:36.281296 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:08:36.281254 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:11.442505 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:11.442447 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:11.540282 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:11.540247 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:11.637587 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:11.637549 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:11.792020 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:11.791970 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:11.891340 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:11.891300 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:12.232742 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:12.232656 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:12.333332 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:12.333292 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:22.491374 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:22.491338 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:22.592333 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:22.592296 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:22.669055 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:22.669015 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:23.343438 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:23.343410 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:23.462051 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:23.462012 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:26.662095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:26.662063 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:26.759653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:26.759619 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:09:26.860123 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:09:26.860092 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:15.227476 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:15.227440 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:15.294887 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:15.294855 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:18.648538 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:18.648506 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:18.748446 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:18.748413 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:18.850179 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:18.850139 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:41.308754 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:41.308702 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:41.433646 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:41.433612 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:48.941803 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:48.941765 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:49.284653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:49.284620 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:49.490730 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:49.490698 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:50.008402 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.008373 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:50.134475 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.134440 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:50.261218 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.261120 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-8kj5j"] Sep 17 19:10:50.265481 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.265456 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" Sep 17 19:10:50.275512 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.275489 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Sep 17 19:10:50.275624 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.275596 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Sep 17 19:10:50.276077 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.276058 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Sep 17 19:10:50.276077 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.276071 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Sep 17 19:10:50.276720 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.276706 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-c4b9h\"" Sep 17 19:10:50.292332 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.292303 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-8kj5j"] Sep 17 19:10:50.319586 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.319530 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:50.356592 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.356559 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw4rh\" (UniqueName: \"kubernetes.io/projected/bd9096dc-2af2-422e-9d34-40694bddbfb4-kube-api-access-mw4rh\") pod \"keda-admission-6c4d864b86-8kj5j\" (UID: \"bd9096dc-2af2-422e-9d34-40694bddbfb4\") " pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" Sep 17 19:10:50.356721 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.356599 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bd9096dc-2af2-422e-9d34-40694bddbfb4-certificates\") pod \"keda-admission-6c4d864b86-8kj5j\" (UID: \"bd9096dc-2af2-422e-9d34-40694bddbfb4\") " pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" Sep 17 19:10:50.457690 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.457653 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw4rh\" (UniqueName: \"kubernetes.io/projected/bd9096dc-2af2-422e-9d34-40694bddbfb4-kube-api-access-mw4rh\") pod \"keda-admission-6c4d864b86-8kj5j\" (UID: \"bd9096dc-2af2-422e-9d34-40694bddbfb4\") " pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" Sep 17 19:10:50.457690 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.457694 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bd9096dc-2af2-422e-9d34-40694bddbfb4-certificates\") pod \"keda-admission-6c4d864b86-8kj5j\" (UID: \"bd9096dc-2af2-422e-9d34-40694bddbfb4\") " pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" Sep 17 19:10:50.460388 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.460351 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bd9096dc-2af2-422e-9d34-40694bddbfb4-certificates\") pod \"keda-admission-6c4d864b86-8kj5j\" (UID: \"bd9096dc-2af2-422e-9d34-40694bddbfb4\") " pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" Sep 17 19:10:50.475302 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.475268 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw4rh\" (UniqueName: \"kubernetes.io/projected/bd9096dc-2af2-422e-9d34-40694bddbfb4-kube-api-access-mw4rh\") pod \"keda-admission-6c4d864b86-8kj5j\" (UID: \"bd9096dc-2af2-422e-9d34-40694bddbfb4\") " pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" Sep 17 19:10:50.571278 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.571228 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:50.577262 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.577228 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" Sep 17 19:10:50.764517 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.764485 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:50.874096 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:50.874073 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-8kj5j"] Sep 17 19:10:50.876781 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:10:50.876744 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd9096dc_2af2_422e_9d34_40694bddbfb4.slice/crio-59de553b968e9ae72a1a9401507eb13ee1f363a50672bc0877b5cfcb1f51f69b WatchSource:0}: Error finding container 59de553b968e9ae72a1a9401507eb13ee1f363a50672bc0877b5cfcb1f51f69b: Status 404 returned error can't find the container with id 59de553b968e9ae72a1a9401507eb13ee1f363a50672bc0877b5cfcb1f51f69b Sep 17 19:10:51.151334 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:51.151247 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:51.399956 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:51.399914 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:51.489150 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:51.489060 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" event={"ID":"bd9096dc-2af2-422e-9d34-40694bddbfb4","Type":"ContainerStarted","Data":"59de553b968e9ae72a1a9401507eb13ee1f363a50672bc0877b5cfcb1f51f69b"} Sep 17 19:10:51.552757 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:51.552720 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:51.680723 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:51.680681 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:10:53.498102 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:53.498012 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" event={"ID":"bd9096dc-2af2-422e-9d34-40694bddbfb4","Type":"ContainerStarted","Data":"8fc4cadfa1f9710547b301aa65478ddcd6ecc0507f0aa004eeec3221f92df1aa"} Sep 17 19:10:53.498466 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:53.498158 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" Sep 17 19:10:53.516650 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:10:53.516600 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" podStartSLOduration=1.341493231 podStartE2EDuration="3.516581572s" totalImagesPullingTime="2.175088341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:10:53.515419797 +0000 UTC m=+257.469041201" watchObservedRunningTime="2026-09-17 19:10:53.516581572 +0000 UTC m=+257.470202930" Sep 17 19:11:00.578186 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:11:00.578087 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:11:04.237057 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:11:04.237013 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:11:14.503824 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:11:14.503796 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-8kj5j" Sep 17 19:11:36.400816 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:11:36.400780 2552 kubelet.go:1648] "Image garbage collection succeeded" Sep 17 19:11:39.560256 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:11:39.560214 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:11:40.691851 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:11:40.691814 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:06.557708 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.557647 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:06.634888 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.634853 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:06.638971 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.638945 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-66576f5db6-2mbkk"] Sep 17 19:12:06.642243 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.642223 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:06.647111 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.647083 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Sep 17 19:12:06.648059 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.648035 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Sep 17 19:12:06.648059 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.648056 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Sep 17 19:12:06.664634 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.664596 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-qz78l\"" Sep 17 19:12:06.683619 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.683583 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-66576f5db6-2mbkk"] Sep 17 19:12:06.698229 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.698169 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-cert\") pod \"kserve-controller-manager-66576f5db6-2mbkk\" (UID: \"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06\") " pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:06.698229 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.698230 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9n8r\" (UniqueName: \"kubernetes.io/projected/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-kube-api-access-w9n8r\") pod \"kserve-controller-manager-66576f5db6-2mbkk\" (UID: \"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06\") " pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:06.772226 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.772190 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:06.799424 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.799380 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-cert\") pod \"kserve-controller-manager-66576f5db6-2mbkk\" (UID: \"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06\") " pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:06.799424 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.799426 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9n8r\" (UniqueName: \"kubernetes.io/projected/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-kube-api-access-w9n8r\") pod \"kserve-controller-manager-66576f5db6-2mbkk\" (UID: \"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06\") " pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:06.799631 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:12:06.799538 2552 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Sep 17 19:12:06.799631 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:12:06.799619 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-cert podName:dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06 nodeName:}" failed. No retries permitted until 2026-09-17 19:12:07.299596875 +0000 UTC m=+331.253218215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-cert") pod "kserve-controller-manager-66576f5db6-2mbkk" (UID: "dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06") : secret "kserve-webhook-server-cert" not found Sep 17 19:12:06.811125 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.811055 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9n8r\" (UniqueName: \"kubernetes.io/projected/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-kube-api-access-w9n8r\") pod \"kserve-controller-manager-66576f5db6-2mbkk\" (UID: \"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06\") " pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:06.860786 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:06.860737 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:07.071768 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:07.071664 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:07.303420 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:07.303373 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-cert\") pod \"kserve-controller-manager-66576f5db6-2mbkk\" (UID: \"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06\") " pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:07.306082 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:07.306057 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-cert\") pod \"kserve-controller-manager-66576f5db6-2mbkk\" (UID: \"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06\") " pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:07.552958 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:07.552893 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:07.586932 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:07.586874 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:07.687925 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:07.687862 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:07.788230 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:07.788194 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:07.828077 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:07.828036 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-66576f5db6-2mbkk"] Sep 17 19:12:07.830975 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:12:07.830946 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfc1b1d6_cdfc_48bc_9c8f_ac9f0192ce06.slice/crio-20ed668d738b59ac928e0a31c3d0c0e5813ba600ad4122bc96107071bcfa73bc WatchSource:0}: Error finding container 20ed668d738b59ac928e0a31c3d0c0e5813ba600ad4122bc96107071bcfa73bc: Status 404 returned error can't find the container with id 20ed668d738b59ac928e0a31c3d0c0e5813ba600ad4122bc96107071bcfa73bc Sep 17 19:12:07.832684 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:07.832661 2552 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 19:12:08.772232 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:08.772193 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" event={"ID":"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06","Type":"ContainerStarted","Data":"20ed668d738b59ac928e0a31c3d0c0e5813ba600ad4122bc96107071bcfa73bc"} Sep 17 19:12:10.780376 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:10.780339 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" event={"ID":"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06","Type":"ContainerStarted","Data":"78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7"} Sep 17 19:12:10.780800 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:10.780447 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:10.797333 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:10.797279 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" podStartSLOduration=2.419145259 podStartE2EDuration="4.797260124s" totalImagesPullingTime="2.378114865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:12:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:12:10.796599123 +0000 UTC m=+334.750220494" watchObservedRunningTime="2026-09-17 19:12:10.797260124 +0000 UTC m=+334.750881482" Sep 17 19:12:33.034181 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:33.034099 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:33.095729 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:33.095695 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:38.903858 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:38.903824 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:41.789613 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:41.789582 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:50.054635 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.054598 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-66576f5db6-2mbkk"] Sep 17 19:12:50.055114 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.054792 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" podUID="dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06" containerName="manager" containerID="cri-o://78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7" gracePeriod=10 Sep 17 19:12:50.078744 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.078718 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-66576f5db6-wmgkk"] Sep 17 19:12:50.080434 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.080414 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" Sep 17 19:12:50.093558 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.093530 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-66576f5db6-wmgkk"] Sep 17 19:12:50.104853 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.104827 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/635348b2-c621-4fb7-a1f4-955986d0897f-cert\") pod \"kserve-controller-manager-66576f5db6-wmgkk\" (UID: \"635348b2-c621-4fb7-a1f4-955986d0897f\") " pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" Sep 17 19:12:50.104999 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.104888 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7d8x\" (UniqueName: \"kubernetes.io/projected/635348b2-c621-4fb7-a1f4-955986d0897f-kube-api-access-t7d8x\") pod \"kserve-controller-manager-66576f5db6-wmgkk\" (UID: \"635348b2-c621-4fb7-a1f4-955986d0897f\") " pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" Sep 17 19:12:50.205556 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.205509 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7d8x\" (UniqueName: \"kubernetes.io/projected/635348b2-c621-4fb7-a1f4-955986d0897f-kube-api-access-t7d8x\") pod \"kserve-controller-manager-66576f5db6-wmgkk\" (UID: \"635348b2-c621-4fb7-a1f4-955986d0897f\") " pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" Sep 17 19:12:50.205720 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.205573 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/635348b2-c621-4fb7-a1f4-955986d0897f-cert\") pod \"kserve-controller-manager-66576f5db6-wmgkk\" (UID: \"635348b2-c621-4fb7-a1f4-955986d0897f\") " pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" Sep 17 19:12:50.208072 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.208046 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/635348b2-c621-4fb7-a1f4-955986d0897f-cert\") pod \"kserve-controller-manager-66576f5db6-wmgkk\" (UID: \"635348b2-c621-4fb7-a1f4-955986d0897f\") " pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" Sep 17 19:12:50.214808 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.214781 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7d8x\" (UniqueName: \"kubernetes.io/projected/635348b2-c621-4fb7-a1f4-955986d0897f-kube-api-access-t7d8x\") pod \"kserve-controller-manager-66576f5db6-wmgkk\" (UID: \"635348b2-c621-4fb7-a1f4-955986d0897f\") " pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" Sep 17 19:12:50.289477 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.289454 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:50.408019 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.407931 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-cert\") pod \"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06\" (UID: \"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06\") " Sep 17 19:12:50.408019 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.408006 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9n8r\" (UniqueName: \"kubernetes.io/projected/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-kube-api-access-w9n8r\") pod \"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06\" (UID: \"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06\") " Sep 17 19:12:50.410241 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.410205 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-cert" (OuterVolumeSpecName: "cert") pod "dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06" (UID: "dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:12:50.410364 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.410296 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-kube-api-access-w9n8r" (OuterVolumeSpecName: "kube-api-access-w9n8r") pod "dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06" (UID: "dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06"). InnerVolumeSpecName "kube-api-access-w9n8r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 19:12:50.426482 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.426452 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" Sep 17 19:12:50.508611 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.508575 2552 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-cert\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:12:50.508611 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.508606 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9n8r\" (UniqueName: \"kubernetes.io/projected/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06-kube-api-access-w9n8r\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:12:50.708544 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.708457 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-66576f5db6-wmgkk"] Sep 17 19:12:50.712668 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:12:50.712635 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod635348b2_c621_4fb7_a1f4_955986d0897f.slice/crio-be55102eda5da3c712a6bc00384a83f2567e7bbb8393942282d9c12107ef1335 WatchSource:0}: Error finding container be55102eda5da3c712a6bc00384a83f2567e7bbb8393942282d9c12107ef1335: Status 404 returned error can't find the container with id be55102eda5da3c712a6bc00384a83f2567e7bbb8393942282d9c12107ef1335 Sep 17 19:12:50.928042 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.928009 2552 generic.go:356] "Generic (PLEG): container finished" podID="dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06" containerID="78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7" exitCode=0 Sep 17 19:12:50.928209 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.928055 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" Sep 17 19:12:50.928209 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.928081 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" event={"ID":"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06","Type":"ContainerDied","Data":"78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7"} Sep 17 19:12:50.928209 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.928127 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66576f5db6-2mbkk" event={"ID":"dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06","Type":"ContainerDied","Data":"20ed668d738b59ac928e0a31c3d0c0e5813ba600ad4122bc96107071bcfa73bc"} Sep 17 19:12:50.928209 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.928154 2552 scope.go:117] "RemoveContainer" containerID="78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7" Sep 17 19:12:50.929336 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.929296 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" event={"ID":"635348b2-c621-4fb7-a1f4-955986d0897f","Type":"ContainerStarted","Data":"be55102eda5da3c712a6bc00384a83f2567e7bbb8393942282d9c12107ef1335"} Sep 17 19:12:50.936699 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.936682 2552 scope.go:117] "RemoveContainer" containerID="78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7" Sep 17 19:12:50.937033 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:12:50.937012 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7\": container with ID starting with 78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7 not found: ID does not exist" containerID="78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7" Sep 17 19:12:50.937120 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.937044 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7"} err="failed to get container status \"78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7\": rpc error: code = NotFound desc = could not find container \"78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7\": container with ID starting with 78e1d165e730398b84cf2cfcf01abbae0f7a933732e1b6fd29c5388fa8c4d5c7 not found: ID does not exist" Sep 17 19:12:50.945134 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.945112 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-66576f5db6-2mbkk"] Sep 17 19:12:50.947667 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:50.947646 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-66576f5db6-2mbkk"] Sep 17 19:12:51.935395 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:51.935361 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" event={"ID":"635348b2-c621-4fb7-a1f4-955986d0897f","Type":"ContainerStarted","Data":"ed154bd5ec0db859dc25624b41eeb597c996528ce2827cb2b4e84aa6be00d01c"} Sep 17 19:12:51.935781 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:51.935469 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" Sep 17 19:12:51.953176 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:51.953134 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" podStartSLOduration=1.5120789079999999 podStartE2EDuration="1.953118052s" totalImagesPullingTime="441.039144ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:12:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:12:51.952219756 +0000 UTC m=+375.905841116" watchObservedRunningTime="2026-09-17 19:12:51.953118052 +0000 UTC m=+375.906739390" Sep 17 19:12:52.515599 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:52.515566 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06" path="/var/lib/kubelet/pods/dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06/volumes" Sep 17 19:12:53.888494 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:53.888461 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:12:53.948827 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:12:53.948787 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:00.814346 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:00.814309 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:00.921024 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:00.920990 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:11.435561 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:11.435514 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:11.495661 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:11.495624 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:11.560137 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:11.560111 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:16.713341 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:16.713303 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:16.768236 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:16.768197 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:18.563920 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:18.563864 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:18.624075 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:18.624038 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:22.944984 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:22.944958 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-66576f5db6-wmgkk" Sep 17 19:13:24.239395 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.239360 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:24.292744 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.292711 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-7b954ffc95-dbzz9"] Sep 17 19:13:24.293108 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.293093 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06" containerName="manager" Sep 17 19:13:24.293173 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.293112 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06" containerName="manager" Sep 17 19:13:24.293225 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.293194 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfc1b1d6-cdfc-48bc-9c8f-ac9f0192ce06" containerName="manager" Sep 17 19:13:24.296295 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.296276 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-dbzz9" Sep 17 19:13:24.300409 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.300389 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\"" Sep 17 19:13:24.300574 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.300555 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-kfswf\"" Sep 17 19:13:24.311064 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.311040 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7b966d7984-cfmsv"] Sep 17 19:13:24.313934 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.313916 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:24.318732 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.318710 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\"" Sep 17 19:13:24.318872 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.318854 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-zk5lx\"" Sep 17 19:13:24.319086 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.319069 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\"" Sep 17 19:13:24.320810 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.320792 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-dbzz9"] Sep 17 19:13:24.334343 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.334319 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-cfmsv"] Sep 17 19:13:24.360876 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.360829 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:24.460739 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.460690 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:24.463654 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.463622 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdfa09dc-3717-4dba-99a4-a3b97484e3b8-metrics-certs\") pod \"odh-model-controller-7b966d7984-cfmsv\" (UID: \"cdfa09dc-3717-4dba-99a4-a3b97484e3b8\") " pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:24.463796 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.463660 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9zx5\" (UniqueName: \"kubernetes.io/projected/71c602b9-85bc-4f7a-8384-d9c647cd6e56-kube-api-access-b9zx5\") pod \"model-serving-api-7b954ffc95-dbzz9\" (UID: \"71c602b9-85bc-4f7a-8384-d9c647cd6e56\") " pod="kserve/model-serving-api-7b954ffc95-dbzz9" Sep 17 19:13:24.463796 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.463721 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71c602b9-85bc-4f7a-8384-d9c647cd6e56-tls-certs\") pod \"model-serving-api-7b954ffc95-dbzz9\" (UID: \"71c602b9-85bc-4f7a-8384-d9c647cd6e56\") " pod="kserve/model-serving-api-7b954ffc95-dbzz9" Sep 17 19:13:24.463796 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.463758 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b76wp\" (UniqueName: \"kubernetes.io/projected/cdfa09dc-3717-4dba-99a4-a3b97484e3b8-kube-api-access-b76wp\") pod \"odh-model-controller-7b966d7984-cfmsv\" (UID: \"cdfa09dc-3717-4dba-99a4-a3b97484e3b8\") " pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:24.463975 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.463850 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdfa09dc-3717-4dba-99a4-a3b97484e3b8-cert\") pod \"odh-model-controller-7b966d7984-cfmsv\" (UID: \"cdfa09dc-3717-4dba-99a4-a3b97484e3b8\") " pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:24.564808 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.564759 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdfa09dc-3717-4dba-99a4-a3b97484e3b8-metrics-certs\") pod \"odh-model-controller-7b966d7984-cfmsv\" (UID: \"cdfa09dc-3717-4dba-99a4-a3b97484e3b8\") " pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:24.565028 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.564823 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9zx5\" (UniqueName: \"kubernetes.io/projected/71c602b9-85bc-4f7a-8384-d9c647cd6e56-kube-api-access-b9zx5\") pod \"model-serving-api-7b954ffc95-dbzz9\" (UID: \"71c602b9-85bc-4f7a-8384-d9c647cd6e56\") " pod="kserve/model-serving-api-7b954ffc95-dbzz9" Sep 17 19:13:24.565028 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.564861 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71c602b9-85bc-4f7a-8384-d9c647cd6e56-tls-certs\") pod \"model-serving-api-7b954ffc95-dbzz9\" (UID: \"71c602b9-85bc-4f7a-8384-d9c647cd6e56\") " pod="kserve/model-serving-api-7b954ffc95-dbzz9" Sep 17 19:13:24.565028 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.564894 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b76wp\" (UniqueName: \"kubernetes.io/projected/cdfa09dc-3717-4dba-99a4-a3b97484e3b8-kube-api-access-b76wp\") pod \"odh-model-controller-7b966d7984-cfmsv\" (UID: \"cdfa09dc-3717-4dba-99a4-a3b97484e3b8\") " pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:24.565028 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:13:24.564923 2552 secret.go:189] Couldn't get secret kserve/odh-model-controller-metrics-tls: secret "odh-model-controller-metrics-tls" not found Sep 17 19:13:24.565028 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.564972 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdfa09dc-3717-4dba-99a4-a3b97484e3b8-cert\") pod \"odh-model-controller-7b966d7984-cfmsv\" (UID: \"cdfa09dc-3717-4dba-99a4-a3b97484e3b8\") " pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:24.565028 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:13:24.565010 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdfa09dc-3717-4dba-99a4-a3b97484e3b8-metrics-certs podName:cdfa09dc-3717-4dba-99a4-a3b97484e3b8 nodeName:}" failed. No retries permitted until 2026-09-17 19:13:25.06498651 +0000 UTC m=+409.018607870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdfa09dc-3717-4dba-99a4-a3b97484e3b8-metrics-certs") pod "odh-model-controller-7b966d7984-cfmsv" (UID: "cdfa09dc-3717-4dba-99a4-a3b97484e3b8") : secret "odh-model-controller-metrics-tls" not found Sep 17 19:13:24.567792 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.567734 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71c602b9-85bc-4f7a-8384-d9c647cd6e56-tls-certs\") pod \"model-serving-api-7b954ffc95-dbzz9\" (UID: \"71c602b9-85bc-4f7a-8384-d9c647cd6e56\") " pod="kserve/model-serving-api-7b954ffc95-dbzz9" Sep 17 19:13:24.567944 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.567796 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cdfa09dc-3717-4dba-99a4-a3b97484e3b8-cert\") pod \"odh-model-controller-7b966d7984-cfmsv\" (UID: \"cdfa09dc-3717-4dba-99a4-a3b97484e3b8\") " pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:24.567944 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.567863 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:24.576967 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.576939 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b76wp\" (UniqueName: \"kubernetes.io/projected/cdfa09dc-3717-4dba-99a4-a3b97484e3b8-kube-api-access-b76wp\") pod \"odh-model-controller-7b966d7984-cfmsv\" (UID: \"cdfa09dc-3717-4dba-99a4-a3b97484e3b8\") " pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:24.577233 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.577214 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9zx5\" (UniqueName: \"kubernetes.io/projected/71c602b9-85bc-4f7a-8384-d9c647cd6e56-kube-api-access-b9zx5\") pod \"model-serving-api-7b954ffc95-dbzz9\" (UID: \"71c602b9-85bc-4f7a-8384-d9c647cd6e56\") " pod="kserve/model-serving-api-7b954ffc95-dbzz9" Sep 17 19:13:24.607174 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.607144 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-7b954ffc95-dbzz9" Sep 17 19:13:24.890170 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.890090 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-7b954ffc95-dbzz9"] Sep 17 19:13:24.893273 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:13:24.893250 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71c602b9_85bc_4f7a_8384_d9c647cd6e56.slice/crio-2e57f1a929e45ab915c371235dce00d5ce952f3f103da1c1183d452c29d56223 WatchSource:0}: Error finding container 2e57f1a929e45ab915c371235dce00d5ce952f3f103da1c1183d452c29d56223: Status 404 returned error can't find the container with id 2e57f1a929e45ab915c371235dce00d5ce952f3f103da1c1183d452c29d56223 Sep 17 19:13:24.938384 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:24.938344 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:25.052627 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:25.052593 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-dbzz9" event={"ID":"71c602b9-85bc-4f7a-8384-d9c647cd6e56","Type":"ContainerStarted","Data":"2e57f1a929e45ab915c371235dce00d5ce952f3f103da1c1183d452c29d56223"} Sep 17 19:13:25.075498 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:25.075455 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdfa09dc-3717-4dba-99a4-a3b97484e3b8-metrics-certs\") pod \"odh-model-controller-7b966d7984-cfmsv\" (UID: \"cdfa09dc-3717-4dba-99a4-a3b97484e3b8\") " pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:25.077925 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:25.077890 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdfa09dc-3717-4dba-99a4-a3b97484e3b8-metrics-certs\") pod \"odh-model-controller-7b966d7984-cfmsv\" (UID: \"cdfa09dc-3717-4dba-99a4-a3b97484e3b8\") " pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:25.226394 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:25.226308 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:25.501790 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:25.501577 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7b966d7984-cfmsv"] Sep 17 19:13:25.505129 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:13:25.505096 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdfa09dc_3717_4dba_99a4_a3b97484e3b8.slice/crio-9e7c802bf9ef0115da7c49abf73ab65a34e42827a45aca119e9f0550f35fffff WatchSource:0}: Error finding container 9e7c802bf9ef0115da7c49abf73ab65a34e42827a45aca119e9f0550f35fffff: Status 404 returned error can't find the container with id 9e7c802bf9ef0115da7c49abf73ab65a34e42827a45aca119e9f0550f35fffff Sep 17 19:13:25.573669 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:25.573633 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:25.673757 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:25.673723 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:13:26.059220 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:26.059181 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-cfmsv" event={"ID":"cdfa09dc-3717-4dba-99a4-a3b97484e3b8","Type":"ContainerStarted","Data":"9e7c802bf9ef0115da7c49abf73ab65a34e42827a45aca119e9f0550f35fffff"} Sep 17 19:13:29.073296 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:29.073256 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-7b954ffc95-dbzz9" event={"ID":"71c602b9-85bc-4f7a-8384-d9c647cd6e56","Type":"ContainerStarted","Data":"78bc4d68eba417d4dbfa1d72647c867c561b75a494544d003fa24c3aa3deed2a"} Sep 17 19:13:29.073758 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:29.073325 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-7b954ffc95-dbzz9" Sep 17 19:13:29.074777 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:29.074755 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7b966d7984-cfmsv" event={"ID":"cdfa09dc-3717-4dba-99a4-a3b97484e3b8","Type":"ContainerStarted","Data":"5ec33bebf1c2e70a26d2fc84dab2a26735a1a5045e11226bfbd5b9f157cb597a"} Sep 17 19:13:29.074890 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:29.074807 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:29.094023 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:29.093973 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-7b954ffc95-dbzz9" podStartSLOduration=1.569095489 podStartE2EDuration="5.093955146s" totalImagesPullingTime="3.524859657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:13:29.091864658 +0000 UTC m=+413.045486017" watchObservedRunningTime="2026-09-17 19:13:29.093955146 +0000 UTC m=+413.047576504" Sep 17 19:13:29.113549 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:29.113497 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/odh-model-controller-7b966d7984-cfmsv" podStartSLOduration=2.157547558 podStartE2EDuration="5.113479771s" totalImagesPullingTime="2.955932213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:13:29.112009507 +0000 UTC m=+413.065630867" watchObservedRunningTime="2026-09-17 19:13:29.113479771 +0000 UTC m=+413.067101156" Sep 17 19:13:40.079701 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:40.079670 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7b966d7984-cfmsv" Sep 17 19:13:40.081720 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:13:40.081703 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-7b954ffc95-dbzz9" Sep 17 19:15:00.825828 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:00.825795 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:15:00.914484 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:00.914440 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:15:13.026739 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.026679 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:15:13.216737 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.216688 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:15:13.343102 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.343015 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:15:13.498887 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.498850 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:15:13.603512 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.603417 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:15:13.699233 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.699152 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:15:13.729795 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.729762 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c"] Sep 17 19:15:13.732967 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.732950 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.735666 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.735610 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Sep 17 19:15:13.735832 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.735808 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Sep 17 19:15:13.735966 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.735896 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Sep 17 19:15:13.736048 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.735974 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Sep 17 19:15:13.736048 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.735977 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tmb8q\"" Sep 17 19:15:13.736048 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.736006 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 19:15:13.736868 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.736851 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Sep 17 19:15:13.744095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.744065 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c"] Sep 17 19:15:13.792988 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.792939 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e3518db-1e4b-4d82-9b99-141bbeb792eb-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.792988 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.792985 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.793228 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.793033 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.793228 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.793055 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e3518db-1e4b-4d82-9b99-141bbeb792eb-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.793228 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.793082 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cnbk\" (UniqueName: \"kubernetes.io/projected/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kube-api-access-5cnbk\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.801678 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.801645 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:15:13.894133 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.894035 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cnbk\" (UniqueName: \"kubernetes.io/projected/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kube-api-access-5cnbk\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.894133 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.894102 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e3518db-1e4b-4d82-9b99-141bbeb792eb-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.894133 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.894129 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.894434 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.894190 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.894627 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.894586 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e3518db-1e4b-4d82-9b99-141bbeb792eb-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.894721 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:15:13.894278 2552 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-serving-cert: secret "isvc-sklearn-graph-2-predictor-serving-cert" not found Sep 17 19:15:13.894721 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.894694 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.894721 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:15:13.894713 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e3518db-1e4b-4d82-9b99-141bbeb792eb-proxy-tls podName:8e3518db-1e4b-4d82-9b99-141bbeb792eb nodeName:}" failed. No retries permitted until 2026-09-17 19:15:14.394690417 +0000 UTC m=+518.348311768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/8e3518db-1e4b-4d82-9b99-141bbeb792eb-proxy-tls") pod "isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" (UID: "8e3518db-1e4b-4d82-9b99-141bbeb792eb") : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found Sep 17 19:15:13.894986 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.894966 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.895180 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.895163 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e3518db-1e4b-4d82-9b99-141bbeb792eb-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.905452 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.905418 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cnbk\" (UniqueName: \"kubernetes.io/projected/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kube-api-access-5cnbk\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:13.987312 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:13.987243 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:15:14.091651 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:14.091608 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:15:14.399384 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:14.399326 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e3518db-1e4b-4d82-9b99-141bbeb792eb-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:14.402012 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:14.401991 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e3518db-1e4b-4d82-9b99-141bbeb792eb-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:14.645249 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:14.645213 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:14.923964 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:14.923935 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c"] Sep 17 19:15:14.926704 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:15:14.926661 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e3518db_1e4b_4d82_9b99_141bbeb792eb.slice/crio-b3dbca35fd1561020d435567cd5b532a1683a317715d29c0ad9b10fb959cf417 WatchSource:0}: Error finding container b3dbca35fd1561020d435567cd5b532a1683a317715d29c0ad9b10fb959cf417: Status 404 returned error can't find the container with id b3dbca35fd1561020d435567cd5b532a1683a317715d29c0ad9b10fb959cf417 Sep 17 19:15:15.461496 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:15.461455 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" event={"ID":"8e3518db-1e4b-4d82-9b99-141bbeb792eb","Type":"ContainerStarted","Data":"b3dbca35fd1561020d435567cd5b532a1683a317715d29c0ad9b10fb959cf417"} Sep 17 19:15:18.477458 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:18.477414 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" event={"ID":"8e3518db-1e4b-4d82-9b99-141bbeb792eb","Type":"ContainerStarted","Data":"16841285c51b7d144d053421bd0e232d76c0b44c9206b1b78165845a1c7bb028"} Sep 17 19:15:22.495894 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:22.495861 2552 generic.go:356] "Generic (PLEG): container finished" podID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerID="16841285c51b7d144d053421bd0e232d76c0b44c9206b1b78165845a1c7bb028" exitCode=0 Sep 17 19:15:22.496274 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:22.495938 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" event={"ID":"8e3518db-1e4b-4d82-9b99-141bbeb792eb","Type":"ContainerDied","Data":"16841285c51b7d144d053421bd0e232d76c0b44c9206b1b78165845a1c7bb028"} Sep 17 19:15:36.573880 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:36.573840 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" event={"ID":"8e3518db-1e4b-4d82-9b99-141bbeb792eb","Type":"ContainerStarted","Data":"8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30"} Sep 17 19:15:38.584195 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:38.584163 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" event={"ID":"8e3518db-1e4b-4d82-9b99-141bbeb792eb","Type":"ContainerStarted","Data":"e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24"} Sep 17 19:15:38.584630 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:38.584388 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:38.584630 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:38.584447 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:38.585711 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:38.585683 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Sep 17 19:15:38.605468 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:38.605411 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podStartSLOduration=3.543763495 podStartE2EDuration="25.605393839s" totalImagesPullingTime="18.386707146s" totalInitContainerRuntime="3.674923198s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:15:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:15:38.604198011 +0000 UTC m=+542.557819372" watchObservedRunningTime="2026-09-17 19:15:38.605393839 +0000 UTC m=+542.559015197" Sep 17 19:15:39.588011 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:39.587974 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Sep 17 19:15:44.593041 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:44.593011 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:15:44.593572 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:44.593548 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Sep 17 19:15:54.593595 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:15:54.593553 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Sep 17 19:16:04.594272 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:04.594233 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Sep 17 19:16:14.477827 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:14.477789 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:16:14.538485 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:14.538448 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:16:14.594035 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:14.593996 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Sep 17 19:16:24.593739 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:24.593699 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Sep 17 19:16:33.003949 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.003897 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm"] Sep 17 19:16:33.006106 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.006085 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:16:33.008036 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.008006 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:16:33.008408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.008392 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-53780-serving-cert\"" Sep 17 19:16:33.008602 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.008583 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-53780-kube-rbac-proxy-sar-config\"" Sep 17 19:16:33.015012 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.014686 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm"] Sep 17 19:16:33.038778 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.038740 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9f9a986e-02ed-404e-b22e-f948abc89d28-proxy-tls\") pod \"switch-graph-53780-7d9f4b6dc5-mgpmm\" (UID: \"9f9a986e-02ed-404e-b22e-f948abc89d28\") " pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:16:33.039003 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.038811 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f9a986e-02ed-404e-b22e-f948abc89d28-openshift-service-ca-bundle\") pod \"switch-graph-53780-7d9f4b6dc5-mgpmm\" (UID: \"9f9a986e-02ed-404e-b22e-f948abc89d28\") " pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:16:33.108319 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.108280 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:16:33.139574 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.139541 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9f9a986e-02ed-404e-b22e-f948abc89d28-proxy-tls\") pod \"switch-graph-53780-7d9f4b6dc5-mgpmm\" (UID: \"9f9a986e-02ed-404e-b22e-f948abc89d28\") " pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:16:33.139761 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.139590 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f9a986e-02ed-404e-b22e-f948abc89d28-openshift-service-ca-bundle\") pod \"switch-graph-53780-7d9f4b6dc5-mgpmm\" (UID: \"9f9a986e-02ed-404e-b22e-f948abc89d28\") " pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:16:33.140302 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.140276 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f9a986e-02ed-404e-b22e-f948abc89d28-openshift-service-ca-bundle\") pod \"switch-graph-53780-7d9f4b6dc5-mgpmm\" (UID: \"9f9a986e-02ed-404e-b22e-f948abc89d28\") " pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:16:33.142323 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.142305 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9f9a986e-02ed-404e-b22e-f948abc89d28-proxy-tls\") pod \"switch-graph-53780-7d9f4b6dc5-mgpmm\" (UID: \"9f9a986e-02ed-404e-b22e-f948abc89d28\") " pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:16:33.319549 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.319501 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:16:33.588448 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.588365 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm"] Sep 17 19:16:33.592080 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:16:33.592040 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f9a986e_02ed_404e_b22e_f948abc89d28.slice/crio-5de8debb39e068ff777d4be3687a12894c04bacb89d7635fde07cc729604dae4 WatchSource:0}: Error finding container 5de8debb39e068ff777d4be3687a12894c04bacb89d7635fde07cc729604dae4: Status 404 returned error can't find the container with id 5de8debb39e068ff777d4be3687a12894c04bacb89d7635fde07cc729604dae4 Sep 17 19:16:33.767678 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.767643 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:16:33.796321 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.796291 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" event={"ID":"9f9a986e-02ed-404e-b22e-f948abc89d28","Type":"ContainerStarted","Data":"5de8debb39e068ff777d4be3687a12894c04bacb89d7635fde07cc729604dae4"} Sep 17 19:16:33.848288 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:33.848194 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:16:34.593589 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:34.593533 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Sep 17 19:16:35.806332 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:35.806294 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" event={"ID":"9f9a986e-02ed-404e-b22e-f948abc89d28","Type":"ContainerStarted","Data":"b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863"} Sep 17 19:16:35.806785 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:35.806399 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:16:41.815471 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:41.815443 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:16:41.832582 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:41.832528 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" podStartSLOduration=7.788378397 podStartE2EDuration="9.83251303s" totalImagesPullingTime="2.044134633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:16:35.824813924 +0000 UTC m=+599.778435281" watchObservedRunningTime="2026-09-17 19:16:41.83251303 +0000 UTC m=+605.786134441" Sep 17 19:16:43.980333 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:43.980301 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm"] Sep 17 19:16:43.980726 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:43.980509 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" podUID="9f9a986e-02ed-404e-b22e-f948abc89d28" containerName="switch-graph-53780" containerID="cri-o://b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863" gracePeriod=30 Sep 17 19:16:44.017374 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:44.017340 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:16:44.593557 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:44.593513 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.21:8080: connect: connection refused" Sep 17 19:16:45.479266 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:45.479219 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:16:46.814165 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:46.814103 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" podUID="9f9a986e-02ed-404e-b22e-f948abc89d28" containerName="switch-graph-53780" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:16:51.814712 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:51.814665 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" podUID="9f9a986e-02ed-404e-b22e-f948abc89d28" containerName="switch-graph-53780" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:16:54.521012 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:54.520985 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:16:56.814417 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:56.814373 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" podUID="9f9a986e-02ed-404e-b22e-f948abc89d28" containerName="switch-graph-53780" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:16:56.814793 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:16:56.814482 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:17:01.814342 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:01.814254 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" podUID="9f9a986e-02ed-404e-b22e-f948abc89d28" containerName="switch-graph-53780" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:17:06.814408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:06.814363 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" podUID="9f9a986e-02ed-404e-b22e-f948abc89d28" containerName="switch-graph-53780" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:17:11.814124 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:11.814067 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" podUID="9f9a986e-02ed-404e-b22e-f948abc89d28" containerName="switch-graph-53780" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:17:14.136624 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.136597 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:17:14.267183 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.267153 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9f9a986e-02ed-404e-b22e-f948abc89d28-proxy-tls\") pod \"9f9a986e-02ed-404e-b22e-f948abc89d28\" (UID: \"9f9a986e-02ed-404e-b22e-f948abc89d28\") " Sep 17 19:17:14.267332 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.267240 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f9a986e-02ed-404e-b22e-f948abc89d28-openshift-service-ca-bundle\") pod \"9f9a986e-02ed-404e-b22e-f948abc89d28\" (UID: \"9f9a986e-02ed-404e-b22e-f948abc89d28\") " Sep 17 19:17:14.267619 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.267593 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f9a986e-02ed-404e-b22e-f948abc89d28-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "9f9a986e-02ed-404e-b22e-f948abc89d28" (UID: "9f9a986e-02ed-404e-b22e-f948abc89d28"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:17:14.269580 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.269549 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f9a986e-02ed-404e-b22e-f948abc89d28-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9f9a986e-02ed-404e-b22e-f948abc89d28" (UID: "9f9a986e-02ed-404e-b22e-f948abc89d28"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:17:14.368439 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.368400 2552 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9f9a986e-02ed-404e-b22e-f948abc89d28-proxy-tls\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:17:14.368439 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.368430 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f9a986e-02ed-404e-b22e-f948abc89d28-openshift-service-ca-bundle\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:17:14.949654 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.949564 2552 generic.go:356] "Generic (PLEG): container finished" podID="9f9a986e-02ed-404e-b22e-f948abc89d28" containerID="b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863" exitCode=0 Sep 17 19:17:14.949654 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.949600 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" event={"ID":"9f9a986e-02ed-404e-b22e-f948abc89d28","Type":"ContainerDied","Data":"b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863"} Sep 17 19:17:14.949654 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.949620 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" Sep 17 19:17:14.949939 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.949679 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm" event={"ID":"9f9a986e-02ed-404e-b22e-f948abc89d28","Type":"ContainerDied","Data":"5de8debb39e068ff777d4be3687a12894c04bacb89d7635fde07cc729604dae4"} Sep 17 19:17:14.949939 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.949703 2552 scope.go:117] "RemoveContainer" containerID="b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863" Sep 17 19:17:14.957889 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.957873 2552 scope.go:117] "RemoveContainer" containerID="b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863" Sep 17 19:17:14.958200 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:17:14.958176 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863\": container with ID starting with b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863 not found: ID does not exist" containerID="b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863" Sep 17 19:17:14.958279 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.958206 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863"} err="failed to get container status \"b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863\": rpc error: code = NotFound desc = could not find container \"b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863\": container with ID starting with b96a0cc3827c1afefcf0bd128600be82e554c11f1556c20b3a981b69a6a7f863 not found: ID does not exist" Sep 17 19:17:14.968232 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.968206 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm"] Sep 17 19:17:14.971970 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:14.971950 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-53780-7d9f4b6dc5-mgpmm"] Sep 17 19:17:16.514811 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:16.514768 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f9a986e-02ed-404e-b22e-f948abc89d28" path="/var/lib/kubelet/pods/9f9a986e-02ed-404e-b22e-f948abc89d28/volumes" Sep 17 19:17:24.422297 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:24.422260 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:17:33.003803 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.003765 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7"] Sep 17 19:17:33.004223 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.004206 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f9a986e-02ed-404e-b22e-f948abc89d28" containerName="switch-graph-53780" Sep 17 19:17:33.004265 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.004227 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f9a986e-02ed-404e-b22e-f948abc89d28" containerName="switch-graph-53780" Sep 17 19:17:33.004320 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.004309 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f9a986e-02ed-404e-b22e-f948abc89d28" containerName="switch-graph-53780" Sep 17 19:17:33.007324 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.007303 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:17:33.009004 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.008975 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7d0ea76-c69e-4b67-ad30-016d58ade526-openshift-service-ca-bundle\") pod \"model-chainer-5bd8dbddfc-rbjq7\" (UID: \"b7d0ea76-c69e-4b67-ad30-016d58ade526\") " pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:17:33.009140 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.009054 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7d0ea76-c69e-4b67-ad30-016d58ade526-proxy-tls\") pod \"model-chainer-5bd8dbddfc-rbjq7\" (UID: \"b7d0ea76-c69e-4b67-ad30-016d58ade526\") " pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:17:33.009653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.009631 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:17:33.009771 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.009751 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Sep 17 19:17:33.009824 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.009782 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\"" Sep 17 19:17:33.087772 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.087733 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7"] Sep 17 19:17:33.109766 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.109729 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7d0ea76-c69e-4b67-ad30-016d58ade526-openshift-service-ca-bundle\") pod \"model-chainer-5bd8dbddfc-rbjq7\" (UID: \"b7d0ea76-c69e-4b67-ad30-016d58ade526\") " pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:17:33.109766 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.109763 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7d0ea76-c69e-4b67-ad30-016d58ade526-proxy-tls\") pod \"model-chainer-5bd8dbddfc-rbjq7\" (UID: \"b7d0ea76-c69e-4b67-ad30-016d58ade526\") " pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:17:33.110033 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:17:33.109889 2552 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found Sep 17 19:17:33.110033 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:17:33.109983 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7d0ea76-c69e-4b67-ad30-016d58ade526-proxy-tls podName:b7d0ea76-c69e-4b67-ad30-016d58ade526 nodeName:}" failed. No retries permitted until 2026-09-17 19:17:33.60996533 +0000 UTC m=+657.563586667 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/b7d0ea76-c69e-4b67-ad30-016d58ade526-proxy-tls") pod "model-chainer-5bd8dbddfc-rbjq7" (UID: "b7d0ea76-c69e-4b67-ad30-016d58ade526") : secret "model-chainer-serving-cert" not found Sep 17 19:17:33.110549 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.110527 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7d0ea76-c69e-4b67-ad30-016d58ade526-openshift-service-ca-bundle\") pod \"model-chainer-5bd8dbddfc-rbjq7\" (UID: \"b7d0ea76-c69e-4b67-ad30-016d58ade526\") " pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:17:33.307883 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.307845 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:17:33.614182 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.614079 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7d0ea76-c69e-4b67-ad30-016d58ade526-proxy-tls\") pod \"model-chainer-5bd8dbddfc-rbjq7\" (UID: \"b7d0ea76-c69e-4b67-ad30-016d58ade526\") " pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:17:33.616786 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.616750 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7d0ea76-c69e-4b67-ad30-016d58ade526-proxy-tls\") pod \"model-chainer-5bd8dbddfc-rbjq7\" (UID: \"b7d0ea76-c69e-4b67-ad30-016d58ade526\") " pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:17:33.618524 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.618505 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:17:33.893205 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:33.893175 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7"] Sep 17 19:17:33.895756 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:17:33.895725 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d0ea76_c69e_4b67_ad30_016d58ade526.slice/crio-01d8725872f038a4075dc4feb81c1f28ed43ac2eb32d6c573dcfd39907641617 WatchSource:0}: Error finding container 01d8725872f038a4075dc4feb81c1f28ed43ac2eb32d6c573dcfd39907641617: Status 404 returned error can't find the container with id 01d8725872f038a4075dc4feb81c1f28ed43ac2eb32d6c573dcfd39907641617 Sep 17 19:17:34.030976 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:34.030941 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" event={"ID":"b7d0ea76-c69e-4b67-ad30-016d58ade526","Type":"ContainerStarted","Data":"6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008"} Sep 17 19:17:34.031444 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:34.030995 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" event={"ID":"b7d0ea76-c69e-4b67-ad30-016d58ade526","Type":"ContainerStarted","Data":"01d8725872f038a4075dc4feb81c1f28ed43ac2eb32d6c573dcfd39907641617"} Sep 17 19:17:34.031444 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:34.031030 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:17:34.049465 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:34.049409 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" podStartSLOduration=2.049390104 podStartE2EDuration="2.049390104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:17:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:17:34.048373425 +0000 UTC m=+658.001994783" watchObservedRunningTime="2026-09-17 19:17:34.049390104 +0000 UTC m=+658.003011463" Sep 17 19:17:40.041292 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:40.041264 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:17:44.561297 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:44.561262 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7"] Sep 17 19:17:44.561742 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:44.561496 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" podUID="b7d0ea76-c69e-4b67-ad30-016d58ade526" containerName="model-chainer" containerID="cri-o://6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008" gracePeriod=30 Sep 17 19:17:44.587610 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:44.587577 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:17:44.741467 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:44.741428 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:17:44.784884 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:44.784852 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c"] Sep 17 19:17:44.785121 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:44.785100 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" containerID="cri-o://8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30" gracePeriod=30 Sep 17 19:17:44.785200 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:44.785148 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kube-rbac-proxy" containerID="cri-o://e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24" gracePeriod=30 Sep 17 19:17:44.840497 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:44.840419 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:17:44.910408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:44.910364 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:17:44.999789 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:44.999750 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:17:45.041075 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:45.041036 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" podUID="b7d0ea76-c69e-4b67-ad30-016d58ade526" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:17:45.074333 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:45.074299 2552 generic.go:356] "Generic (PLEG): container finished" podID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerID="e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24" exitCode=2 Sep 17 19:17:45.074492 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:45.074376 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" event={"ID":"8e3518db-1e4b-4d82-9b99-141bbeb792eb","Type":"ContainerDied","Data":"e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24"} Sep 17 19:17:45.096975 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:45.096867 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:17:45.279481 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:45.279441 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:17:49.589223 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:49.589177 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.21:8643/healthz\": dial tcp 10.133.0.21:8643: connect: connection refused" Sep 17 19:17:50.033861 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.033837 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:17:50.039578 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.039542 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" podUID="b7d0ea76-c69e-4b67-ad30-016d58ade526" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:17:50.094779 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.094742 2552 generic.go:356] "Generic (PLEG): container finished" podID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerID="8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30" exitCode=0 Sep 17 19:17:50.094964 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.094808 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" Sep 17 19:17:50.094964 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.094810 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" event={"ID":"8e3518db-1e4b-4d82-9b99-141bbeb792eb","Type":"ContainerDied","Data":"8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30"} Sep 17 19:17:50.094964 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.094933 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c" event={"ID":"8e3518db-1e4b-4d82-9b99-141bbeb792eb","Type":"ContainerDied","Data":"b3dbca35fd1561020d435567cd5b532a1683a317715d29c0ad9b10fb959cf417"} Sep 17 19:17:50.095097 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.094980 2552 scope.go:117] "RemoveContainer" containerID="e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24" Sep 17 19:17:50.105106 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.105085 2552 scope.go:117] "RemoveContainer" containerID="8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30" Sep 17 19:17:50.112872 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.112852 2552 scope.go:117] "RemoveContainer" containerID="16841285c51b7d144d053421bd0e232d76c0b44c9206b1b78165845a1c7bb028" Sep 17 19:17:50.120405 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.120386 2552 scope.go:117] "RemoveContainer" containerID="e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24" Sep 17 19:17:50.120670 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:17:50.120651 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24\": container with ID starting with e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24 not found: ID does not exist" containerID="e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24" Sep 17 19:17:50.120721 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.120683 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24"} err="failed to get container status \"e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24\": rpc error: code = NotFound desc = could not find container \"e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24\": container with ID starting with e74685a11908302179ea47da9e52de5e37449da99927487bc53bb72f1b5aea24 not found: ID does not exist" Sep 17 19:17:50.120721 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.120701 2552 scope.go:117] "RemoveContainer" containerID="8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30" Sep 17 19:17:50.120950 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:17:50.120932 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30\": container with ID starting with 8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30 not found: ID does not exist" containerID="8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30" Sep 17 19:17:50.121001 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.120955 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30"} err="failed to get container status \"8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30\": rpc error: code = NotFound desc = could not find container \"8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30\": container with ID starting with 8d32c1a21386d6e86da75349d97ef9fc5da01d1b5c4f3a5d527dd97c6fc1bb30 not found: ID does not exist" Sep 17 19:17:50.121001 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.120968 2552 scope.go:117] "RemoveContainer" containerID="16841285c51b7d144d053421bd0e232d76c0b44c9206b1b78165845a1c7bb028" Sep 17 19:17:50.121240 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:17:50.121223 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16841285c51b7d144d053421bd0e232d76c0b44c9206b1b78165845a1c7bb028\": container with ID starting with 16841285c51b7d144d053421bd0e232d76c0b44c9206b1b78165845a1c7bb028 not found: ID does not exist" containerID="16841285c51b7d144d053421bd0e232d76c0b44c9206b1b78165845a1c7bb028" Sep 17 19:17:50.121277 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.121245 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16841285c51b7d144d053421bd0e232d76c0b44c9206b1b78165845a1c7bb028"} err="failed to get container status \"16841285c51b7d144d053421bd0e232d76c0b44c9206b1b78165845a1c7bb028\": rpc error: code = NotFound desc = could not find container \"16841285c51b7d144d053421bd0e232d76c0b44c9206b1b78165845a1c7bb028\": container with ID starting with 16841285c51b7d144d053421bd0e232d76c0b44c9206b1b78165845a1c7bb028 not found: ID does not exist" Sep 17 19:17:50.123502 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.123481 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cnbk\" (UniqueName: \"kubernetes.io/projected/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kube-api-access-5cnbk\") pod \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " Sep 17 19:17:50.123577 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.123518 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e3518db-1e4b-4d82-9b99-141bbeb792eb-cabundle-cert\") pod \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " Sep 17 19:17:50.123577 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.123549 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e3518db-1e4b-4d82-9b99-141bbeb792eb-proxy-tls\") pod \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " Sep 17 19:17:50.123653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.123580 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kserve-provision-location\") pod \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " Sep 17 19:17:50.123653 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.123646 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kube-rbac-proxy-sar-config\") pod \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\" (UID: \"8e3518db-1e4b-4d82-9b99-141bbeb792eb\") " Sep 17 19:17:50.123970 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.123947 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e3518db-1e4b-4d82-9b99-141bbeb792eb-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "8e3518db-1e4b-4d82-9b99-141bbeb792eb" (UID: "8e3518db-1e4b-4d82-9b99-141bbeb792eb"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:17:50.124032 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.123970 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8e3518db-1e4b-4d82-9b99-141bbeb792eb" (UID: "8e3518db-1e4b-4d82-9b99-141bbeb792eb"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 19:17:50.124088 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.124042 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "8e3518db-1e4b-4d82-9b99-141bbeb792eb" (UID: "8e3518db-1e4b-4d82-9b99-141bbeb792eb"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:17:50.125739 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.125717 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kube-api-access-5cnbk" (OuterVolumeSpecName: "kube-api-access-5cnbk") pod "8e3518db-1e4b-4d82-9b99-141bbeb792eb" (UID: "8e3518db-1e4b-4d82-9b99-141bbeb792eb"). InnerVolumeSpecName "kube-api-access-5cnbk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 19:17:50.125826 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.125741 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e3518db-1e4b-4d82-9b99-141bbeb792eb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8e3518db-1e4b-4d82-9b99-141bbeb792eb" (UID: "8e3518db-1e4b-4d82-9b99-141bbeb792eb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:17:50.225206 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.225120 2552 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8e3518db-1e4b-4d82-9b99-141bbeb792eb-proxy-tls\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.225206 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.225150 2552 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kserve-provision-location\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.225206 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.225160 2552 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kube-rbac-proxy-sar-config\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.225206 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.225169 2552 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cnbk\" (UniqueName: \"kubernetes.io/projected/8e3518db-1e4b-4d82-9b99-141bbeb792eb-kube-api-access-5cnbk\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.225206 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.225178 2552 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8e3518db-1e4b-4d82-9b99-141bbeb792eb-cabundle-cert\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:17:50.420156 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.420120 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c"] Sep 17 19:17:50.422974 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.422946 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-74b67d48cc-h9g4c"] Sep 17 19:17:50.515227 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:50.515199 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" path="/var/lib/kubelet/pods/8e3518db-1e4b-4d82-9b99-141bbeb792eb/volumes" Sep 17 19:17:54.222591 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.222550 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s"] Sep 17 19:17:54.223090 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.223043 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="storage-initializer" Sep 17 19:17:54.223090 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.223063 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="storage-initializer" Sep 17 19:17:54.223090 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.223075 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kube-rbac-proxy" Sep 17 19:17:54.223090 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.223082 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kube-rbac-proxy" Sep 17 19:17:54.223281 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.223099 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" Sep 17 19:17:54.223281 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.223115 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" Sep 17 19:17:54.223281 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.223238 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kserve-container" Sep 17 19:17:54.223281 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.223250 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e3518db-1e4b-4d82-9b99-141bbeb792eb" containerName="kube-rbac-proxy" Sep 17 19:17:54.224656 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.224632 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:17:54.227661 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.227642 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:17:54.230332 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.230312 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-8c825-serving-cert\"" Sep 17 19:17:54.230433 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.230412 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-8c825-kube-rbac-proxy-sar-config\"" Sep 17 19:17:54.235062 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.235042 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s"] Sep 17 19:17:54.254544 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.254513 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4c65c45f-c7b2-4795-8584-845159742841-proxy-tls\") pod \"switch-graph-8c825-84586659f5-6hs9s\" (UID: \"4c65c45f-c7b2-4795-8584-845159742841\") " pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:17:54.254676 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.254567 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c65c45f-c7b2-4795-8584-845159742841-openshift-service-ca-bundle\") pod \"switch-graph-8c825-84586659f5-6hs9s\" (UID: \"4c65c45f-c7b2-4795-8584-845159742841\") " pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:17:54.355021 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.354976 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4c65c45f-c7b2-4795-8584-845159742841-proxy-tls\") pod \"switch-graph-8c825-84586659f5-6hs9s\" (UID: \"4c65c45f-c7b2-4795-8584-845159742841\") " pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:17:54.355211 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.355030 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c65c45f-c7b2-4795-8584-845159742841-openshift-service-ca-bundle\") pod \"switch-graph-8c825-84586659f5-6hs9s\" (UID: \"4c65c45f-c7b2-4795-8584-845159742841\") " pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:17:54.355211 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:17:54.355166 2552 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-8c825-serving-cert: secret "switch-graph-8c825-serving-cert" not found Sep 17 19:17:54.355325 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:17:54.355251 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c65c45f-c7b2-4795-8584-845159742841-proxy-tls podName:4c65c45f-c7b2-4795-8584-845159742841 nodeName:}" failed. No retries permitted until 2026-09-17 19:17:54.855229418 +0000 UTC m=+678.808850773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/4c65c45f-c7b2-4795-8584-845159742841-proxy-tls") pod "switch-graph-8c825-84586659f5-6hs9s" (UID: "4c65c45f-c7b2-4795-8584-845159742841") : secret "switch-graph-8c825-serving-cert" not found Sep 17 19:17:54.355596 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.355577 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c65c45f-c7b2-4795-8584-845159742841-openshift-service-ca-bundle\") pod \"switch-graph-8c825-84586659f5-6hs9s\" (UID: \"4c65c45f-c7b2-4795-8584-845159742841\") " pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:17:54.449320 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.449283 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:17:54.860116 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.860064 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4c65c45f-c7b2-4795-8584-845159742841-proxy-tls\") pod \"switch-graph-8c825-84586659f5-6hs9s\" (UID: \"4c65c45f-c7b2-4795-8584-845159742841\") " pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:17:54.862795 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:54.862768 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4c65c45f-c7b2-4795-8584-845159742841-proxy-tls\") pod \"switch-graph-8c825-84586659f5-6hs9s\" (UID: \"4c65c45f-c7b2-4795-8584-845159742841\") " pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:17:55.039423 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:55.039378 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" podUID="b7d0ea76-c69e-4b67-ad30-016d58ade526" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:17:55.039629 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:55.039492 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:17:55.138310 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:55.138209 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:17:55.409757 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:55.409716 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s"] Sep 17 19:17:55.412439 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:17:55.412399 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c65c45f_c7b2_4795_8584_845159742841.slice/crio-00ceedab1fcc98740cec7cfb02b53c8ebfd6db69a320f9275cc433fb623d1235 WatchSource:0}: Error finding container 00ceedab1fcc98740cec7cfb02b53c8ebfd6db69a320f9275cc433fb623d1235: Status 404 returned error can't find the container with id 00ceedab1fcc98740cec7cfb02b53c8ebfd6db69a320f9275cc433fb623d1235 Sep 17 19:17:56.120862 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:56.120820 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" event={"ID":"4c65c45f-c7b2-4795-8584-845159742841","Type":"ContainerStarted","Data":"d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd"} Sep 17 19:17:56.120862 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:56.120872 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" event={"ID":"4c65c45f-c7b2-4795-8584-845159742841","Type":"ContainerStarted","Data":"00ceedab1fcc98740cec7cfb02b53c8ebfd6db69a320f9275cc433fb623d1235"} Sep 17 19:17:56.121132 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:56.120970 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:17:56.137861 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:17:56.137814 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" podStartSLOduration=2.137798196 podStartE2EDuration="2.137798196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:17:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:17:56.136189761 +0000 UTC m=+680.089811114" watchObservedRunningTime="2026-09-17 19:17:56.137798196 +0000 UTC m=+680.091419555" Sep 17 19:18:00.039655 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:00.039619 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" podUID="b7d0ea76-c69e-4b67-ad30-016d58ade526" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:18:02.130827 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:02.130787 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:18:05.040182 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:05.040135 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" podUID="b7d0ea76-c69e-4b67-ad30-016d58ade526" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:18:10.039558 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:10.039515 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" podUID="b7d0ea76-c69e-4b67-ad30-016d58ade526" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:18:14.755626 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:14.755601 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:18:14.809564 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:14.809527 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7d0ea76-c69e-4b67-ad30-016d58ade526-openshift-service-ca-bundle\") pod \"b7d0ea76-c69e-4b67-ad30-016d58ade526\" (UID: \"b7d0ea76-c69e-4b67-ad30-016d58ade526\") " Sep 17 19:18:14.809711 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:14.809580 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7d0ea76-c69e-4b67-ad30-016d58ade526-proxy-tls\") pod \"b7d0ea76-c69e-4b67-ad30-016d58ade526\" (UID: \"b7d0ea76-c69e-4b67-ad30-016d58ade526\") " Sep 17 19:18:14.809960 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:14.809938 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7d0ea76-c69e-4b67-ad30-016d58ade526-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b7d0ea76-c69e-4b67-ad30-016d58ade526" (UID: "b7d0ea76-c69e-4b67-ad30-016d58ade526"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:18:14.811808 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:14.811785 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7d0ea76-c69e-4b67-ad30-016d58ade526-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b7d0ea76-c69e-4b67-ad30-016d58ade526" (UID: "b7d0ea76-c69e-4b67-ad30-016d58ade526"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:18:14.910685 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:14.910647 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b7d0ea76-c69e-4b67-ad30-016d58ade526-openshift-service-ca-bundle\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:18:14.910685 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:14.910677 2552 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b7d0ea76-c69e-4b67-ad30-016d58ade526-proxy-tls\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:18:15.195223 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:15.195140 2552 generic.go:356] "Generic (PLEG): container finished" podID="b7d0ea76-c69e-4b67-ad30-016d58ade526" containerID="6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008" exitCode=137 Sep 17 19:18:15.195223 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:15.195186 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" Sep 17 19:18:15.195427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:15.195224 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" event={"ID":"b7d0ea76-c69e-4b67-ad30-016d58ade526","Type":"ContainerDied","Data":"6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008"} Sep 17 19:18:15.195427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:15.195276 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7" event={"ID":"b7d0ea76-c69e-4b67-ad30-016d58ade526","Type":"ContainerDied","Data":"01d8725872f038a4075dc4feb81c1f28ed43ac2eb32d6c573dcfd39907641617"} Sep 17 19:18:15.195427 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:15.195298 2552 scope.go:117] "RemoveContainer" containerID="6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008" Sep 17 19:18:15.203772 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:15.203757 2552 scope.go:117] "RemoveContainer" containerID="6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008" Sep 17 19:18:15.204100 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:18:15.204072 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008\": container with ID starting with 6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008 not found: ID does not exist" containerID="6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008" Sep 17 19:18:15.204203 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:15.204108 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008"} err="failed to get container status \"6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008\": rpc error: code = NotFound desc = could not find container \"6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008\": container with ID starting with 6f974406c0e2ff9778e2503a36fc8bcfd697462e7d988b6c5b52a7e873296008 not found: ID does not exist" Sep 17 19:18:15.216217 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:15.216185 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7"] Sep 17 19:18:15.218153 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:15.218121 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-5bd8dbddfc-rbjq7"] Sep 17 19:18:16.514816 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:16.514781 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7d0ea76-c69e-4b67-ad30-016d58ade526" path="/var/lib/kubelet/pods/b7d0ea76-c69e-4b67-ad30-016d58ade526/volumes" Sep 17 19:18:54.775648 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:54.775612 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq"] Sep 17 19:18:54.776171 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:54.775926 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7d0ea76-c69e-4b67-ad30-016d58ade526" containerName="model-chainer" Sep 17 19:18:54.776171 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:54.775943 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7d0ea76-c69e-4b67-ad30-016d58ade526" containerName="model-chainer" Sep 17 19:18:54.776171 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:54.776011 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7d0ea76-c69e-4b67-ad30-016d58ade526" containerName="model-chainer" Sep 17 19:18:54.780269 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:54.780246 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:18:54.782717 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:54.782673 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-fcbf3-serving-cert\"" Sep 17 19:18:54.782850 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:54.782833 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-fcbf3-kube-rbac-proxy-sar-config\"" Sep 17 19:18:54.788824 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:54.788725 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq"] Sep 17 19:18:54.789854 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:54.789828 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:18:54.918176 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:54.918140 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:18:54.922065 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:54.922040 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff5727bb-528b-4064-b0e3-03d65fd163dc-openshift-service-ca-bundle\") pod \"sequence-graph-fcbf3-85495fc98d-vb8sq\" (UID: \"ff5727bb-528b-4064-b0e3-03d65fd163dc\") " pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:18:54.922183 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:54.922092 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff5727bb-528b-4064-b0e3-03d65fd163dc-proxy-tls\") pod \"sequence-graph-fcbf3-85495fc98d-vb8sq\" (UID: \"ff5727bb-528b-4064-b0e3-03d65fd163dc\") " pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:18:55.022584 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:55.022555 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff5727bb-528b-4064-b0e3-03d65fd163dc-openshift-service-ca-bundle\") pod \"sequence-graph-fcbf3-85495fc98d-vb8sq\" (UID: \"ff5727bb-528b-4064-b0e3-03d65fd163dc\") " pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:18:55.022752 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:55.022616 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff5727bb-528b-4064-b0e3-03d65fd163dc-proxy-tls\") pod \"sequence-graph-fcbf3-85495fc98d-vb8sq\" (UID: \"ff5727bb-528b-4064-b0e3-03d65fd163dc\") " pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:18:55.023373 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:55.023349 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff5727bb-528b-4064-b0e3-03d65fd163dc-openshift-service-ca-bundle\") pod \"sequence-graph-fcbf3-85495fc98d-vb8sq\" (UID: \"ff5727bb-528b-4064-b0e3-03d65fd163dc\") " pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:18:55.025239 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:55.025214 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff5727bb-528b-4064-b0e3-03d65fd163dc-proxy-tls\") pod \"sequence-graph-fcbf3-85495fc98d-vb8sq\" (UID: \"ff5727bb-528b-4064-b0e3-03d65fd163dc\") " pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:18:55.091375 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:55.091253 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:18:55.359788 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:55.359645 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq"] Sep 17 19:18:55.362461 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:18:55.362429 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff5727bb_528b_4064_b0e3_03d65fd163dc.slice/crio-3f71708de728d6530f870b0b8c1caef6eebc968a01e014f6c00b90e0201f13a5 WatchSource:0}: Error finding container 3f71708de728d6530f870b0b8c1caef6eebc968a01e014f6c00b90e0201f13a5: Status 404 returned error can't find the container with id 3f71708de728d6530f870b0b8c1caef6eebc968a01e014f6c00b90e0201f13a5 Sep 17 19:18:56.348611 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:56.348580 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" event={"ID":"ff5727bb-528b-4064-b0e3-03d65fd163dc","Type":"ContainerStarted","Data":"0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8"} Sep 17 19:18:56.348611 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:56.348629 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" event={"ID":"ff5727bb-528b-4064-b0e3-03d65fd163dc","Type":"ContainerStarted","Data":"3f71708de728d6530f870b0b8c1caef6eebc968a01e014f6c00b90e0201f13a5"} Sep 17 19:18:56.349158 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:56.348651 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:18:56.365144 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:18:56.365098 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" podStartSLOduration=2.365081741 podStartE2EDuration="2.365081741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:18:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:18:56.364543857 +0000 UTC m=+740.318165215" watchObservedRunningTime="2026-09-17 19:18:56.365081741 +0000 UTC m=+740.318703099" Sep 17 19:19:02.357705 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:19:02.357668 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:19:07.936523 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:19:07.936477 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:22:47.506174 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:22:47.506144 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:26:09.232990 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:09.232889 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s"] Sep 17 19:26:09.233519 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:09.233129 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" podUID="4c65c45f-c7b2-4795-8584-845159742841" containerName="switch-graph-8c825" containerID="cri-o://d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd" gracePeriod=30 Sep 17 19:26:09.283223 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:09.283195 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:26:09.388239 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:09.388201 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:26:09.488878 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:09.488796 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:26:09.581332 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:09.581286 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:26:09.646510 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:09.646473 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:26:09.760084 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:09.760013 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:26:10.148296 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:10.148266 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:26:12.129991 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:12.129952 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" podUID="4c65c45f-c7b2-4795-8584-845159742841" containerName="switch-graph-8c825" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:26:17.129309 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:17.129274 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" podUID="4c65c45f-c7b2-4795-8584-845159742841" containerName="switch-graph-8c825" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:26:22.129580 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:22.129536 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" podUID="4c65c45f-c7b2-4795-8584-845159742841" containerName="switch-graph-8c825" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:26:22.129984 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:22.129693 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:26:27.129293 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:27.129254 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" podUID="4c65c45f-c7b2-4795-8584-845159742841" containerName="switch-graph-8c825" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:26:32.129464 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:32.129426 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" podUID="4c65c45f-c7b2-4795-8584-845159742841" containerName="switch-graph-8c825" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:26:37.129915 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:37.129873 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" podUID="4c65c45f-c7b2-4795-8584-845159742841" containerName="switch-graph-8c825" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:26:39.413120 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:39.413096 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:26:39.506975 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:39.506865 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c65c45f-c7b2-4795-8584-845159742841-openshift-service-ca-bundle\") pod \"4c65c45f-c7b2-4795-8584-845159742841\" (UID: \"4c65c45f-c7b2-4795-8584-845159742841\") " Sep 17 19:26:39.506975 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:39.506966 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4c65c45f-c7b2-4795-8584-845159742841-proxy-tls\") pod \"4c65c45f-c7b2-4795-8584-845159742841\" (UID: \"4c65c45f-c7b2-4795-8584-845159742841\") " Sep 17 19:26:39.507230 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:39.507205 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c65c45f-c7b2-4795-8584-845159742841-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "4c65c45f-c7b2-4795-8584-845159742841" (UID: "4c65c45f-c7b2-4795-8584-845159742841"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:26:39.509139 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:39.509119 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c65c45f-c7b2-4795-8584-845159742841-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4c65c45f-c7b2-4795-8584-845159742841" (UID: "4c65c45f-c7b2-4795-8584-845159742841"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:26:39.608149 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:39.608108 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4c65c45f-c7b2-4795-8584-845159742841-openshift-service-ca-bundle\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:26:39.608149 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:39.608139 2552 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4c65c45f-c7b2-4795-8584-845159742841-proxy-tls\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:26:40.038986 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:40.038956 2552 generic.go:356] "Generic (PLEG): container finished" podID="4c65c45f-c7b2-4795-8584-845159742841" containerID="d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd" exitCode=0 Sep 17 19:26:40.039247 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:40.039000 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" Sep 17 19:26:40.039247 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:40.039006 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" event={"ID":"4c65c45f-c7b2-4795-8584-845159742841","Type":"ContainerDied","Data":"d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd"} Sep 17 19:26:40.039247 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:40.039049 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s" event={"ID":"4c65c45f-c7b2-4795-8584-845159742841","Type":"ContainerDied","Data":"00ceedab1fcc98740cec7cfb02b53c8ebfd6db69a320f9275cc433fb623d1235"} Sep 17 19:26:40.039247 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:40.039072 2552 scope.go:117] "RemoveContainer" containerID="d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd" Sep 17 19:26:40.047571 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:40.047554 2552 scope.go:117] "RemoveContainer" containerID="d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd" Sep 17 19:26:40.047842 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:26:40.047826 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd\": container with ID starting with d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd not found: ID does not exist" containerID="d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd" Sep 17 19:26:40.047887 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:40.047853 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd"} err="failed to get container status \"d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd\": rpc error: code = NotFound desc = could not find container \"d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd\": container with ID starting with d8f7be9d020ebb9137dc7372637470d29bbedae4316af4e650292cc354be5ccd not found: ID does not exist" Sep 17 19:26:40.062851 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:40.061320 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s"] Sep 17 19:26:40.065218 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:40.065199 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-8c825-84586659f5-6hs9s"] Sep 17 19:26:40.514639 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:26:40.514609 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c65c45f-c7b2-4795-8584-845159742841" path="/var/lib/kubelet/pods/4c65c45f-c7b2-4795-8584-845159742841/volumes" Sep 17 19:27:10.396195 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:10.396165 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq"] Sep 17 19:27:10.396626 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:10.396359 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" podUID="ff5727bb-528b-4064-b0e3-03d65fd163dc" containerName="sequence-graph-fcbf3" containerID="cri-o://0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8" gracePeriod=30 Sep 17 19:27:10.439019 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:10.438950 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:10.583302 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:10.583208 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:10.684616 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:10.684513 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:10.780104 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:10.780062 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:10.877806 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:10.877775 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:10.979132 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:10.979048 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:11.077322 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:11.077291 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:12.356785 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:12.356749 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" podUID="ff5727bb-528b-4064-b0e3-03d65fd163dc" containerName="sequence-graph-fcbf3" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:27:17.356619 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:17.356572 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" podUID="ff5727bb-528b-4064-b0e3-03d65fd163dc" containerName="sequence-graph-fcbf3" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:27:19.478707 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.478624 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:19.482273 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.482247 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g"] Sep 17 19:27:19.482543 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.482532 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c65c45f-c7b2-4795-8584-845159742841" containerName="switch-graph-8c825" Sep 17 19:27:19.482586 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.482545 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c65c45f-c7b2-4795-8584-845159742841" containerName="switch-graph-8c825" Sep 17 19:27:19.482620 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.482588 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c65c45f-c7b2-4795-8584-845159742841" containerName="switch-graph-8c825" Sep 17 19:27:19.485534 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.485515 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:27:19.488156 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.488134 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-87c35-serving-cert\"" Sep 17 19:27:19.488603 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.488586 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-87c35-kube-rbac-proxy-sar-config\"" Sep 17 19:27:19.502386 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.502361 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g"] Sep 17 19:27:19.605539 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.605504 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-proxy-tls\") pod \"ensemble-graph-87c35-66784999db-c5w9g\" (UID: \"73ec24fb-4e16-476c-9cf8-6ddd72f83de5\") " pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:27:19.605702 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.605571 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-openshift-service-ca-bundle\") pod \"ensemble-graph-87c35-66784999db-c5w9g\" (UID: \"73ec24fb-4e16-476c-9cf8-6ddd72f83de5\") " pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:27:19.706589 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.706552 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-openshift-service-ca-bundle\") pod \"ensemble-graph-87c35-66784999db-c5w9g\" (UID: \"73ec24fb-4e16-476c-9cf8-6ddd72f83de5\") " pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:27:19.706786 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.706600 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-proxy-tls\") pod \"ensemble-graph-87c35-66784999db-c5w9g\" (UID: \"73ec24fb-4e16-476c-9cf8-6ddd72f83de5\") " pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:27:19.706786 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:27:19.706723 2552 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-87c35-serving-cert: secret "ensemble-graph-87c35-serving-cert" not found Sep 17 19:27:19.706934 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:27:19.706803 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-proxy-tls podName:73ec24fb-4e16-476c-9cf8-6ddd72f83de5 nodeName:}" failed. No retries permitted until 2026-09-17 19:27:20.206774222 +0000 UTC m=+1244.160395567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-proxy-tls") pod "ensemble-graph-87c35-66784999db-c5w9g" (UID: "73ec24fb-4e16-476c-9cf8-6ddd72f83de5") : secret "ensemble-graph-87c35-serving-cert" not found Sep 17 19:27:19.707333 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.707308 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-openshift-service-ca-bundle\") pod \"ensemble-graph-87c35-66784999db-c5w9g\" (UID: \"73ec24fb-4e16-476c-9cf8-6ddd72f83de5\") " pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:27:19.893843 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:19.893807 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:20.211205 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:20.211124 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-proxy-tls\") pod \"ensemble-graph-87c35-66784999db-c5w9g\" (UID: \"73ec24fb-4e16-476c-9cf8-6ddd72f83de5\") " pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:27:20.213587 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:20.213569 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-proxy-tls\") pod \"ensemble-graph-87c35-66784999db-c5w9g\" (UID: \"73ec24fb-4e16-476c-9cf8-6ddd72f83de5\") " pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:27:20.395386 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:20.395353 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:27:20.665012 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:20.664991 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g"] Sep 17 19:27:20.669291 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:27:20.669262 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ec24fb_4e16_476c_9cf8_6ddd72f83de5.slice/crio-826fcf959b0fdf46326931d52d15c0f51309e9ecac99bf409703c995ab217af2 WatchSource:0}: Error finding container 826fcf959b0fdf46326931d52d15c0f51309e9ecac99bf409703c995ab217af2: Status 404 returned error can't find the container with id 826fcf959b0fdf46326931d52d15c0f51309e9ecac99bf409703c995ab217af2 Sep 17 19:27:21.191798 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:21.191767 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" event={"ID":"73ec24fb-4e16-476c-9cf8-6ddd72f83de5","Type":"ContainerStarted","Data":"d48c093b13164745369ca49e8319574d3493417eceffa0fa4549d899fa1e9d01"} Sep 17 19:27:21.191987 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:21.191814 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" event={"ID":"73ec24fb-4e16-476c-9cf8-6ddd72f83de5","Type":"ContainerStarted","Data":"826fcf959b0fdf46326931d52d15c0f51309e9ecac99bf409703c995ab217af2"} Sep 17 19:27:21.191987 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:21.191845 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:27:21.210401 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:21.210352 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" podStartSLOduration=2.210338414 podStartE2EDuration="2.210338414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:27:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:27:21.209750265 +0000 UTC m=+1245.163371659" watchObservedRunningTime="2026-09-17 19:27:21.210338414 +0000 UTC m=+1245.163959773" Sep 17 19:27:22.356344 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:22.356303 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" podUID="ff5727bb-528b-4064-b0e3-03d65fd163dc" containerName="sequence-graph-fcbf3" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:27:22.356724 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:22.356409 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:27:27.201039 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:27.201012 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:27:27.356611 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:27.356570 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" podUID="ff5727bb-528b-4064-b0e3-03d65fd163dc" containerName="sequence-graph-fcbf3" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:27:30.194631 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:30.194593 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g"] Sep 17 19:27:30.197108 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:30.194796 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" podUID="73ec24fb-4e16-476c-9cf8-6ddd72f83de5" containerName="ensemble-graph-87c35" containerID="cri-o://d48c093b13164745369ca49e8319574d3493417eceffa0fa4549d899fa1e9d01" gracePeriod=30 Sep 17 19:27:30.217063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:30.217001 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:30.323843 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:30.323810 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:30.431926 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:30.431872 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:30.506810 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:30.506735 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:30.610593 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:30.610559 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:30.707452 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:30.707405 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:30.819920 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:30.819853 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:27:32.199444 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:32.199409 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" podUID="73ec24fb-4e16-476c-9cf8-6ddd72f83de5" containerName="ensemble-graph-87c35" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:27:32.356152 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:32.356111 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" podUID="ff5727bb-528b-4064-b0e3-03d65fd163dc" containerName="sequence-graph-fcbf3" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:27:37.199492 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:37.199458 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" podUID="73ec24fb-4e16-476c-9cf8-6ddd72f83de5" containerName="ensemble-graph-87c35" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:27:37.356476 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:37.356439 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" podUID="ff5727bb-528b-4064-b0e3-03d65fd163dc" containerName="sequence-graph-fcbf3" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:27:40.534557 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:40.534531 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:27:40.661961 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:40.661859 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff5727bb-528b-4064-b0e3-03d65fd163dc-proxy-tls\") pod \"ff5727bb-528b-4064-b0e3-03d65fd163dc\" (UID: \"ff5727bb-528b-4064-b0e3-03d65fd163dc\") " Sep 17 19:27:40.661961 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:40.661924 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff5727bb-528b-4064-b0e3-03d65fd163dc-openshift-service-ca-bundle\") pod \"ff5727bb-528b-4064-b0e3-03d65fd163dc\" (UID: \"ff5727bb-528b-4064-b0e3-03d65fd163dc\") " Sep 17 19:27:40.662327 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:40.662295 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff5727bb-528b-4064-b0e3-03d65fd163dc-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "ff5727bb-528b-4064-b0e3-03d65fd163dc" (UID: "ff5727bb-528b-4064-b0e3-03d65fd163dc"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:27:40.664093 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:40.664068 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff5727bb-528b-4064-b0e3-03d65fd163dc-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ff5727bb-528b-4064-b0e3-03d65fd163dc" (UID: "ff5727bb-528b-4064-b0e3-03d65fd163dc"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:27:40.763284 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:40.763257 2552 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ff5727bb-528b-4064-b0e3-03d65fd163dc-proxy-tls\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:27:40.763284 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:40.763278 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff5727bb-528b-4064-b0e3-03d65fd163dc-openshift-service-ca-bundle\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:27:41.269146 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:41.269111 2552 generic.go:356] "Generic (PLEG): container finished" podID="ff5727bb-528b-4064-b0e3-03d65fd163dc" containerID="0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8" exitCode=0 Sep 17 19:27:41.269315 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:41.269152 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" Sep 17 19:27:41.269315 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:41.269160 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" event={"ID":"ff5727bb-528b-4064-b0e3-03d65fd163dc","Type":"ContainerDied","Data":"0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8"} Sep 17 19:27:41.269315 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:41.269199 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq" event={"ID":"ff5727bb-528b-4064-b0e3-03d65fd163dc","Type":"ContainerDied","Data":"3f71708de728d6530f870b0b8c1caef6eebc968a01e014f6c00b90e0201f13a5"} Sep 17 19:27:41.269315 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:41.269221 2552 scope.go:117] "RemoveContainer" containerID="0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8" Sep 17 19:27:41.277463 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:41.277445 2552 scope.go:117] "RemoveContainer" containerID="0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8" Sep 17 19:27:41.277701 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:27:41.277685 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8\": container with ID starting with 0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8 not found: ID does not exist" containerID="0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8" Sep 17 19:27:41.277745 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:41.277707 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8"} err="failed to get container status \"0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8\": rpc error: code = NotFound desc = could not find container \"0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8\": container with ID starting with 0ff2bacde74734fcc6eddae26d1fc907fc76cddece092d4881ba59a9723e09c8 not found: ID does not exist" Sep 17 19:27:41.294697 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:41.290540 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq"] Sep 17 19:27:41.296418 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:41.296396 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-fcbf3-85495fc98d-vb8sq"] Sep 17 19:27:42.199611 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:42.199570 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" podUID="73ec24fb-4e16-476c-9cf8-6ddd72f83de5" containerName="ensemble-graph-87c35" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:27:42.200063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:42.199695 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:27:42.514754 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:42.514724 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff5727bb-528b-4064-b0e3-03d65fd163dc" path="/var/lib/kubelet/pods/ff5727bb-528b-4064-b0e3-03d65fd163dc/volumes" Sep 17 19:27:47.199534 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:47.199494 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" podUID="73ec24fb-4e16-476c-9cf8-6ddd72f83de5" containerName="ensemble-graph-87c35" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:27:52.200211 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:52.200176 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" podUID="73ec24fb-4e16-476c-9cf8-6ddd72f83de5" containerName="ensemble-graph-87c35" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:27:57.200355 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:57.200318 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" podUID="73ec24fb-4e16-476c-9cf8-6ddd72f83de5" containerName="ensemble-graph-87c35" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:27:57.479654 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:27:57.479560 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:28:00.339832 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:00.339797 2552 generic.go:356] "Generic (PLEG): container finished" podID="73ec24fb-4e16-476c-9cf8-6ddd72f83de5" containerID="d48c093b13164745369ca49e8319574d3493417eceffa0fa4549d899fa1e9d01" exitCode=0 Sep 17 19:28:00.340193 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:00.339846 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" event={"ID":"73ec24fb-4e16-476c-9cf8-6ddd72f83de5","Type":"ContainerDied","Data":"d48c093b13164745369ca49e8319574d3493417eceffa0fa4549d899fa1e9d01"} Sep 17 19:28:00.833876 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:00.833852 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:28:01.013286 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:01.013260 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-proxy-tls\") pod \"73ec24fb-4e16-476c-9cf8-6ddd72f83de5\" (UID: \"73ec24fb-4e16-476c-9cf8-6ddd72f83de5\") " Sep 17 19:28:01.013454 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:01.013308 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-openshift-service-ca-bundle\") pod \"73ec24fb-4e16-476c-9cf8-6ddd72f83de5\" (UID: \"73ec24fb-4e16-476c-9cf8-6ddd72f83de5\") " Sep 17 19:28:01.013681 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:01.013657 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "73ec24fb-4e16-476c-9cf8-6ddd72f83de5" (UID: "73ec24fb-4e16-476c-9cf8-6ddd72f83de5"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:28:01.015536 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:01.015516 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "73ec24fb-4e16-476c-9cf8-6ddd72f83de5" (UID: "73ec24fb-4e16-476c-9cf8-6ddd72f83de5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:28:01.114567 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:01.114533 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-openshift-service-ca-bundle\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:28:01.114567 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:01.114559 2552 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/73ec24fb-4e16-476c-9cf8-6ddd72f83de5-proxy-tls\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:28:01.345296 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:01.345215 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" event={"ID":"73ec24fb-4e16-476c-9cf8-6ddd72f83de5","Type":"ContainerDied","Data":"826fcf959b0fdf46326931d52d15c0f51309e9ecac99bf409703c995ab217af2"} Sep 17 19:28:01.345296 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:01.345269 2552 scope.go:117] "RemoveContainer" containerID="d48c093b13164745369ca49e8319574d3493417eceffa0fa4549d899fa1e9d01" Sep 17 19:28:01.345741 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:01.345223 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g" Sep 17 19:28:01.365021 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:01.364999 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g"] Sep 17 19:28:01.373297 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:01.373271 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-87c35-66784999db-c5w9g"] Sep 17 19:28:02.515486 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:02.515454 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73ec24fb-4e16-476c-9cf8-6ddd72f83de5" path="/var/lib/kubelet/pods/73ec24fb-4e16-476c-9cf8-6ddd72f83de5/volumes" Sep 17 19:28:20.644128 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.644094 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw"] Sep 17 19:28:20.644522 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.644454 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73ec24fb-4e16-476c-9cf8-6ddd72f83de5" containerName="ensemble-graph-87c35" Sep 17 19:28:20.644522 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.644468 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="73ec24fb-4e16-476c-9cf8-6ddd72f83de5" containerName="ensemble-graph-87c35" Sep 17 19:28:20.644522 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.644486 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff5727bb-528b-4064-b0e3-03d65fd163dc" containerName="sequence-graph-fcbf3" Sep 17 19:28:20.644522 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.644495 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff5727bb-528b-4064-b0e3-03d65fd163dc" containerName="sequence-graph-fcbf3" Sep 17 19:28:20.644649 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.644570 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff5727bb-528b-4064-b0e3-03d65fd163dc" containerName="sequence-graph-fcbf3" Sep 17 19:28:20.644649 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.644583 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="73ec24fb-4e16-476c-9cf8-6ddd72f83de5" containerName="ensemble-graph-87c35" Sep 17 19:28:20.647531 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.647514 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:28:20.650256 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.650233 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-30eba-serving-cert\"" Sep 17 19:28:20.650256 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.650251 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 19:28:20.650526 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.650514 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-30eba-kube-rbac-proxy-sar-config\"" Sep 17 19:28:20.650822 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.650806 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:28:20.651391 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.651372 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tmb8q\"" Sep 17 19:28:20.667752 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.667730 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw"] Sep 17 19:28:20.746649 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.746620 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3faa556c-190c-4e43-abc0-2261d88c5ae4-proxy-tls\") pod \"sequence-graph-30eba-68667c8b8c-qjtfw\" (UID: \"3faa556c-190c-4e43-abc0-2261d88c5ae4\") " pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:28:20.746809 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.746661 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3faa556c-190c-4e43-abc0-2261d88c5ae4-openshift-service-ca-bundle\") pod \"sequence-graph-30eba-68667c8b8c-qjtfw\" (UID: \"3faa556c-190c-4e43-abc0-2261d88c5ae4\") " pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:28:20.847170 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.847143 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3faa556c-190c-4e43-abc0-2261d88c5ae4-openshift-service-ca-bundle\") pod \"sequence-graph-30eba-68667c8b8c-qjtfw\" (UID: \"3faa556c-190c-4e43-abc0-2261d88c5ae4\") " pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:28:20.847339 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.847199 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3faa556c-190c-4e43-abc0-2261d88c5ae4-proxy-tls\") pod \"sequence-graph-30eba-68667c8b8c-qjtfw\" (UID: \"3faa556c-190c-4e43-abc0-2261d88c5ae4\") " pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:28:20.847339 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:28:20.847322 2552 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-30eba-serving-cert: secret "sequence-graph-30eba-serving-cert" not found Sep 17 19:28:20.847421 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:28:20.847401 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3faa556c-190c-4e43-abc0-2261d88c5ae4-proxy-tls podName:3faa556c-190c-4e43-abc0-2261d88c5ae4 nodeName:}" failed. No retries permitted until 2026-09-17 19:28:21.34737907 +0000 UTC m=+1305.301000428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3faa556c-190c-4e43-abc0-2261d88c5ae4-proxy-tls") pod "sequence-graph-30eba-68667c8b8c-qjtfw" (UID: "3faa556c-190c-4e43-abc0-2261d88c5ae4") : secret "sequence-graph-30eba-serving-cert" not found Sep 17 19:28:20.847836 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:20.847819 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3faa556c-190c-4e43-abc0-2261d88c5ae4-openshift-service-ca-bundle\") pod \"sequence-graph-30eba-68667c8b8c-qjtfw\" (UID: \"3faa556c-190c-4e43-abc0-2261d88c5ae4\") " pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:28:21.035220 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:21.035187 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:28:21.352386 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:21.352300 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3faa556c-190c-4e43-abc0-2261d88c5ae4-proxy-tls\") pod \"sequence-graph-30eba-68667c8b8c-qjtfw\" (UID: \"3faa556c-190c-4e43-abc0-2261d88c5ae4\") " pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:28:21.354950 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:21.354896 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3faa556c-190c-4e43-abc0-2261d88c5ae4-proxy-tls\") pod \"sequence-graph-30eba-68667c8b8c-qjtfw\" (UID: \"3faa556c-190c-4e43-abc0-2261d88c5ae4\") " pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:28:21.557400 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:21.557351 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:28:21.825762 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:21.825652 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw"] Sep 17 19:28:21.828447 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:28:21.828422 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3faa556c_190c_4e43_abc0_2261d88c5ae4.slice/crio-fdfc7a65e38f93e026f23c8765359735390d9362d79822db98cb97e591697575 WatchSource:0}: Error finding container fdfc7a65e38f93e026f23c8765359735390d9362d79822db98cb97e591697575: Status 404 returned error can't find the container with id fdfc7a65e38f93e026f23c8765359735390d9362d79822db98cb97e591697575 Sep 17 19:28:22.422860 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:22.422821 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" event={"ID":"3faa556c-190c-4e43-abc0-2261d88c5ae4","Type":"ContainerStarted","Data":"5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc"} Sep 17 19:28:22.423049 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:22.422882 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" event={"ID":"3faa556c-190c-4e43-abc0-2261d88c5ae4","Type":"ContainerStarted","Data":"fdfc7a65e38f93e026f23c8765359735390d9362d79822db98cb97e591697575"} Sep 17 19:28:22.423095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:22.423044 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:28:22.443124 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:22.443082 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" podStartSLOduration=2.443067121 podStartE2EDuration="2.443067121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:28:22.442269986 +0000 UTC m=+1306.395891344" watchObservedRunningTime="2026-09-17 19:28:22.443067121 +0000 UTC m=+1306.396688522" Sep 17 19:28:28.432800 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:28.432763 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:28:31.588063 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:31.588031 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw"] Sep 17 19:28:31.588500 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:31.588267 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" podUID="3faa556c-190c-4e43-abc0-2261d88c5ae4" containerName="sequence-graph-30eba" containerID="cri-o://5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc" gracePeriod=30 Sep 17 19:28:31.609283 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:31.609256 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:28:31.715705 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:31.715671 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:28:31.807812 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:31.807784 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:28:31.892548 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:31.892469 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:28:31.969606 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:31.969572 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:28:32.059408 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:32.059370 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:28:32.309876 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:32.309850 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:28:33.431157 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:33.431119 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" podUID="3faa556c-190c-4e43-abc0-2261d88c5ae4" containerName="sequence-graph-30eba" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:28:38.430251 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:38.430213 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" podUID="3faa556c-190c-4e43-abc0-2261d88c5ae4" containerName="sequence-graph-30eba" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:28:40.448626 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:40.448595 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5"] Sep 17 19:28:40.452590 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:40.452573 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:28:40.455508 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:40.455478 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-31ec7-serving-cert\"" Sep 17 19:28:40.455508 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:40.455499 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-31ec7-kube-rbac-proxy-sar-config\"" Sep 17 19:28:40.458482 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:40.458432 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:28:40.461438 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:40.461417 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5"] Sep 17 19:28:40.500125 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:40.500096 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49c32d92-b8ee-4523-9f73-8afb065c7d18-openshift-service-ca-bundle\") pod \"ensemble-graph-31ec7-7b66875df7-pfqh5\" (UID: \"49c32d92-b8ee-4523-9f73-8afb065c7d18\") " pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:28:40.500267 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:40.500131 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49c32d92-b8ee-4523-9f73-8afb065c7d18-proxy-tls\") pod \"ensemble-graph-31ec7-7b66875df7-pfqh5\" (UID: \"49c32d92-b8ee-4523-9f73-8afb065c7d18\") " pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:28:40.601321 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:40.601282 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49c32d92-b8ee-4523-9f73-8afb065c7d18-openshift-service-ca-bundle\") pod \"ensemble-graph-31ec7-7b66875df7-pfqh5\" (UID: \"49c32d92-b8ee-4523-9f73-8afb065c7d18\") " pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:28:40.601321 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:40.601319 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49c32d92-b8ee-4523-9f73-8afb065c7d18-proxy-tls\") pod \"ensemble-graph-31ec7-7b66875df7-pfqh5\" (UID: \"49c32d92-b8ee-4523-9f73-8afb065c7d18\") " pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:28:40.601537 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:28:40.601409 2552 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-31ec7-serving-cert: secret "ensemble-graph-31ec7-serving-cert" not found Sep 17 19:28:40.601537 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:28:40.601461 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49c32d92-b8ee-4523-9f73-8afb065c7d18-proxy-tls podName:49c32d92-b8ee-4523-9f73-8afb065c7d18 nodeName:}" failed. No retries permitted until 2026-09-17 19:28:41.101446165 +0000 UTC m=+1325.055067502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/49c32d92-b8ee-4523-9f73-8afb065c7d18-proxy-tls") pod "ensemble-graph-31ec7-7b66875df7-pfqh5" (UID: "49c32d92-b8ee-4523-9f73-8afb065c7d18") : secret "ensemble-graph-31ec7-serving-cert" not found Sep 17 19:28:40.601891 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:40.601870 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49c32d92-b8ee-4523-9f73-8afb065c7d18-openshift-service-ca-bundle\") pod \"ensemble-graph-31ec7-7b66875df7-pfqh5\" (UID: \"49c32d92-b8ee-4523-9f73-8afb065c7d18\") " pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:28:40.876826 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:40.876793 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:28:41.106350 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:41.106314 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49c32d92-b8ee-4523-9f73-8afb065c7d18-proxy-tls\") pod \"ensemble-graph-31ec7-7b66875df7-pfqh5\" (UID: \"49c32d92-b8ee-4523-9f73-8afb065c7d18\") " pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:28:41.108782 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:41.108763 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49c32d92-b8ee-4523-9f73-8afb065c7d18-proxy-tls\") pod \"ensemble-graph-31ec7-7b66875df7-pfqh5\" (UID: \"49c32d92-b8ee-4523-9f73-8afb065c7d18\") " pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:28:41.362879 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:41.362844 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:28:41.631294 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:41.631205 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5"] Sep 17 19:28:41.634142 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:28:41.634089 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49c32d92_b8ee_4523_9f73_8afb065c7d18.slice/crio-2f70a253d363d7a178ceff9190c066a1b1c9a02baae69333d501ef44c06370fb WatchSource:0}: Error finding container 2f70a253d363d7a178ceff9190c066a1b1c9a02baae69333d501ef44c06370fb: Status 404 returned error can't find the container with id 2f70a253d363d7a178ceff9190c066a1b1c9a02baae69333d501ef44c06370fb Sep 17 19:28:42.496047 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:42.496011 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" event={"ID":"49c32d92-b8ee-4523-9f73-8afb065c7d18","Type":"ContainerStarted","Data":"d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f"} Sep 17 19:28:42.496221 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:42.496086 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" event={"ID":"49c32d92-b8ee-4523-9f73-8afb065c7d18","Type":"ContainerStarted","Data":"2f70a253d363d7a178ceff9190c066a1b1c9a02baae69333d501ef44c06370fb"} Sep 17 19:28:42.496221 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:42.496135 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:28:42.514970 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:42.514893 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" podStartSLOduration=2.514875857 podStartE2EDuration="2.514875857s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:28:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:28:42.51374407 +0000 UTC m=+1326.467365433" watchObservedRunningTime="2026-09-17 19:28:42.514875857 +0000 UTC m=+1326.468497252" Sep 17 19:28:43.431213 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:43.431178 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" podUID="3faa556c-190c-4e43-abc0-2261d88c5ae4" containerName="sequence-graph-30eba" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:28:43.431641 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:43.431280 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:28:48.430776 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:48.430741 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" podUID="3faa556c-190c-4e43-abc0-2261d88c5ae4" containerName="sequence-graph-30eba" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:28:48.505285 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:48.505257 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:28:53.430500 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:53.430456 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" podUID="3faa556c-190c-4e43-abc0-2261d88c5ae4" containerName="sequence-graph-30eba" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:28:58.430827 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:28:58.430742 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" podUID="3faa556c-190c-4e43-abc0-2261d88c5ae4" containerName="sequence-graph-30eba" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:29:01.729958 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:01.729935 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:29:01.869646 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:01.869565 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3faa556c-190c-4e43-abc0-2261d88c5ae4-openshift-service-ca-bundle\") pod \"3faa556c-190c-4e43-abc0-2261d88c5ae4\" (UID: \"3faa556c-190c-4e43-abc0-2261d88c5ae4\") " Sep 17 19:29:01.869646 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:01.869637 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3faa556c-190c-4e43-abc0-2261d88c5ae4-proxy-tls\") pod \"3faa556c-190c-4e43-abc0-2261d88c5ae4\" (UID: \"3faa556c-190c-4e43-abc0-2261d88c5ae4\") " Sep 17 19:29:01.869949 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:01.869890 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3faa556c-190c-4e43-abc0-2261d88c5ae4-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "3faa556c-190c-4e43-abc0-2261d88c5ae4" (UID: "3faa556c-190c-4e43-abc0-2261d88c5ae4"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:29:01.871777 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:01.871755 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3faa556c-190c-4e43-abc0-2261d88c5ae4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3faa556c-190c-4e43-abc0-2261d88c5ae4" (UID: "3faa556c-190c-4e43-abc0-2261d88c5ae4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:29:01.970040 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:01.970011 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3faa556c-190c-4e43-abc0-2261d88c5ae4-openshift-service-ca-bundle\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:29:01.970040 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:01.970035 2552 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3faa556c-190c-4e43-abc0-2261d88c5ae4-proxy-tls\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:29:02.569357 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:02.569321 2552 generic.go:356] "Generic (PLEG): container finished" podID="3faa556c-190c-4e43-abc0-2261d88c5ae4" containerID="5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc" exitCode=0 Sep 17 19:29:02.569516 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:02.569366 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" Sep 17 19:29:02.569516 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:02.569407 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" event={"ID":"3faa556c-190c-4e43-abc0-2261d88c5ae4","Type":"ContainerDied","Data":"5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc"} Sep 17 19:29:02.569516 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:02.569461 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw" event={"ID":"3faa556c-190c-4e43-abc0-2261d88c5ae4","Type":"ContainerDied","Data":"fdfc7a65e38f93e026f23c8765359735390d9362d79822db98cb97e591697575"} Sep 17 19:29:02.569516 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:02.569488 2552 scope.go:117] "RemoveContainer" containerID="5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc" Sep 17 19:29:02.577611 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:02.577594 2552 scope.go:117] "RemoveContainer" containerID="5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc" Sep 17 19:29:02.577916 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:29:02.577884 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc\": container with ID starting with 5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc not found: ID does not exist" containerID="5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc" Sep 17 19:29:02.577985 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:02.577928 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc"} err="failed to get container status \"5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc\": rpc error: code = NotFound desc = could not find container \"5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc\": container with ID starting with 5af6a3fe212825a8de98b76d435bebc3449c7da6b21de4d459aef386366729bc not found: ID does not exist" Sep 17 19:29:02.586398 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:02.586379 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw"] Sep 17 19:29:02.589884 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:02.589865 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-30eba-68667c8b8c-qjtfw"] Sep 17 19:29:04.515257 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:04.515201 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3faa556c-190c-4e43-abc0-2261d88c5ae4" path="/var/lib/kubelet/pods/3faa556c-190c-4e43-abc0-2261d88c5ae4/volumes" Sep 17 19:29:41.838468 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.838437 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:29:41.841686 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.841660 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l"] Sep 17 19:29:41.842000 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.841984 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3faa556c-190c-4e43-abc0-2261d88c5ae4" containerName="sequence-graph-30eba" Sep 17 19:29:41.842084 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.842002 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3faa556c-190c-4e43-abc0-2261d88c5ae4" containerName="sequence-graph-30eba" Sep 17 19:29:41.842162 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.842096 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3faa556c-190c-4e43-abc0-2261d88c5ae4" containerName="sequence-graph-30eba" Sep 17 19:29:41.845974 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.845944 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:29:41.848354 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.848323 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-proxy-tls\") pod \"sequence-graph-cedb0-c565b857f-8kg7l\" (UID: \"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951\") " pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:29:41.848458 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.848391 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-cedb0-kube-rbac-proxy-sar-config\"" Sep 17 19:29:41.848458 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.848442 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-cedb0-serving-cert\"" Sep 17 19:29:41.848558 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.848442 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-openshift-service-ca-bundle\") pod \"sequence-graph-cedb0-c565b857f-8kg7l\" (UID: \"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951\") " pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:29:41.855880 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.855858 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l"] Sep 17 19:29:41.949751 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.949718 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-proxy-tls\") pod \"sequence-graph-cedb0-c565b857f-8kg7l\" (UID: \"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951\") " pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:29:41.949931 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.949820 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-openshift-service-ca-bundle\") pod \"sequence-graph-cedb0-c565b857f-8kg7l\" (UID: \"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951\") " pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:29:41.949989 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:29:41.949927 2552 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-cedb0-serving-cert: secret "sequence-graph-cedb0-serving-cert" not found Sep 17 19:29:41.950024 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:29:41.949996 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-proxy-tls podName:3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951 nodeName:}" failed. No retries permitted until 2026-09-17 19:29:42.449979894 +0000 UTC m=+1386.403601230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-proxy-tls") pod "sequence-graph-cedb0-c565b857f-8kg7l" (UID: "3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951") : secret "sequence-graph-cedb0-serving-cert" not found Sep 17 19:29:41.950434 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:41.950417 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-openshift-service-ca-bundle\") pod \"sequence-graph-cedb0-c565b857f-8kg7l\" (UID: \"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951\") " pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:29:42.040851 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:42.040802 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:29:42.453423 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:42.453383 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-proxy-tls\") pod \"sequence-graph-cedb0-c565b857f-8kg7l\" (UID: \"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951\") " pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:29:42.455913 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:42.455886 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-proxy-tls\") pod \"sequence-graph-cedb0-c565b857f-8kg7l\" (UID: \"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951\") " pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:29:42.756213 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:42.756127 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:29:43.025707 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:43.025680 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l"] Sep 17 19:29:43.028062 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:29:43.028032 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c2b6c6e_b9de_4cd7_a0b5_e9bf3e922951.slice/crio-d83071cb6532f729200e59105a682cb5361322b947a4080a1734c13a92832d93 WatchSource:0}: Error finding container d83071cb6532f729200e59105a682cb5361322b947a4080a1734c13a92832d93: Status 404 returned error can't find the container with id d83071cb6532f729200e59105a682cb5361322b947a4080a1734c13a92832d93 Sep 17 19:29:43.724229 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:43.724190 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" event={"ID":"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951","Type":"ContainerStarted","Data":"1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8"} Sep 17 19:29:43.724229 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:43.724238 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" event={"ID":"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951","Type":"ContainerStarted","Data":"d83071cb6532f729200e59105a682cb5361322b947a4080a1734c13a92832d93"} Sep 17 19:29:43.724491 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:43.724323 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:29:43.743162 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:43.743108 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" podStartSLOduration=2.743092686 podStartE2EDuration="2.743092686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:29:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:29:43.742087669 +0000 UTC m=+1387.695709025" watchObservedRunningTime="2026-09-17 19:29:43.743092686 +0000 UTC m=+1387.696714044" Sep 17 19:29:49.734139 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:29:49.734095 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:30:29.188073 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:30:29.187988 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:31:24.554985 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:31:24.554948 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:36:55.749974 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:36:55.749865 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5"] Sep 17 19:36:55.750559 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:36:55.750145 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" podUID="49c32d92-b8ee-4523-9f73-8afb065c7d18" containerName="ensemble-graph-31ec7" containerID="cri-o://d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f" gracePeriod=30 Sep 17 19:36:55.798622 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:36:55.798592 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:36:57.698247 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:36:57.698212 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:36:58.503704 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:36:58.503665 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" podUID="49c32d92-b8ee-4523-9f73-8afb065c7d18" containerName="ensemble-graph-31ec7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:37:03.503572 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:03.503532 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" podUID="49c32d92-b8ee-4523-9f73-8afb065c7d18" containerName="ensemble-graph-31ec7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:37:08.504250 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:08.504208 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" podUID="49c32d92-b8ee-4523-9f73-8afb065c7d18" containerName="ensemble-graph-31ec7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:37:08.504668 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:08.504336 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:37:13.503566 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:13.503531 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" podUID="49c32d92-b8ee-4523-9f73-8afb065c7d18" containerName="ensemble-graph-31ec7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:37:18.503853 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:18.503817 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" podUID="49c32d92-b8ee-4523-9f73-8afb065c7d18" containerName="ensemble-graph-31ec7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:37:23.504095 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:23.504047 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" podUID="49c32d92-b8ee-4523-9f73-8afb065c7d18" containerName="ensemble-graph-31ec7" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:37:25.883493 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:25.883470 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:37:25.959778 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:25.959743 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49c32d92-b8ee-4523-9f73-8afb065c7d18-proxy-tls\") pod \"49c32d92-b8ee-4523-9f73-8afb065c7d18\" (UID: \"49c32d92-b8ee-4523-9f73-8afb065c7d18\") " Sep 17 19:37:25.959984 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:25.959805 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49c32d92-b8ee-4523-9f73-8afb065c7d18-openshift-service-ca-bundle\") pod \"49c32d92-b8ee-4523-9f73-8afb065c7d18\" (UID: \"49c32d92-b8ee-4523-9f73-8afb065c7d18\") " Sep 17 19:37:25.960194 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:25.960160 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49c32d92-b8ee-4523-9f73-8afb065c7d18-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "49c32d92-b8ee-4523-9f73-8afb065c7d18" (UID: "49c32d92-b8ee-4523-9f73-8afb065c7d18"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:37:25.962007 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:25.961979 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49c32d92-b8ee-4523-9f73-8afb065c7d18-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "49c32d92-b8ee-4523-9f73-8afb065c7d18" (UID: "49c32d92-b8ee-4523-9f73-8afb065c7d18"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:37:26.060507 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:26.060473 2552 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49c32d92-b8ee-4523-9f73-8afb065c7d18-proxy-tls\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:37:26.060507 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:26.060502 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49c32d92-b8ee-4523-9f73-8afb065c7d18-openshift-service-ca-bundle\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:37:26.418268 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:26.418180 2552 generic.go:356] "Generic (PLEG): container finished" podID="49c32d92-b8ee-4523-9f73-8afb065c7d18" containerID="d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f" exitCode=0 Sep 17 19:37:26.418268 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:26.418244 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" Sep 17 19:37:26.418541 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:26.418269 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" event={"ID":"49c32d92-b8ee-4523-9f73-8afb065c7d18","Type":"ContainerDied","Data":"d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f"} Sep 17 19:37:26.418541 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:26.418349 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5" event={"ID":"49c32d92-b8ee-4523-9f73-8afb065c7d18","Type":"ContainerDied","Data":"2f70a253d363d7a178ceff9190c066a1b1c9a02baae69333d501ef44c06370fb"} Sep 17 19:37:26.418541 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:26.418371 2552 scope.go:117] "RemoveContainer" containerID="d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f" Sep 17 19:37:26.426691 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:26.426676 2552 scope.go:117] "RemoveContainer" containerID="d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f" Sep 17 19:37:26.426971 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:37:26.426951 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f\": container with ID starting with d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f not found: ID does not exist" containerID="d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f" Sep 17 19:37:26.427043 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:26.426978 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f"} err="failed to get container status \"d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f\": rpc error: code = NotFound desc = could not find container \"d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f\": container with ID starting with d7a6fbc5d5e353856846a04a8a17e80064929266dacb456da70324fee8029d8f not found: ID does not exist" Sep 17 19:37:26.438137 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:26.438116 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5"] Sep 17 19:37:26.443525 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:26.443506 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-31ec7-7b66875df7-pfqh5"] Sep 17 19:37:26.514881 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:26.514854 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49c32d92-b8ee-4523-9f73-8afb065c7d18" path="/var/lib/kubelet/pods/49c32d92-b8ee-4523-9f73-8afb065c7d18/volumes" Sep 17 19:37:56.963884 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:56.963848 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l"] Sep 17 19:37:56.964307 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:56.964064 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" podUID="3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" containerName="sequence-graph-cedb0" containerID="cri-o://1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8" gracePeriod=30 Sep 17 19:37:56.993540 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:56.993512 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:37:57.131373 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:57.131344 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:37:57.202496 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:57.202461 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:37:57.303772 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:57.303744 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:37:57.403110 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:57.403075 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:37:57.508854 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:57.508826 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:37:57.934885 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:57.934806 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:37:59.732254 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:37:59.732219 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" podUID="3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" containerName="sequence-graph-cedb0" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:38:04.024068 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:04.024031 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:38:04.733237 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:04.733197 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" podUID="3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" containerName="sequence-graph-cedb0" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:38:05.997393 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:05.997317 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j"] Sep 17 19:38:05.997729 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:05.997580 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49c32d92-b8ee-4523-9f73-8afb065c7d18" containerName="ensemble-graph-31ec7" Sep 17 19:38:05.997729 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:05.997592 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="49c32d92-b8ee-4523-9f73-8afb065c7d18" containerName="ensemble-graph-31ec7" Sep 17 19:38:05.997729 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:05.997662 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="49c32d92-b8ee-4523-9f73-8afb065c7d18" containerName="ensemble-graph-31ec7" Sep 17 19:38:06.000714 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.000687 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:06.003278 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.003253 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-fe2a8-kube-rbac-proxy-sar-config\"" Sep 17 19:38:06.003397 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.003279 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-fe2a8-serving-cert\"" Sep 17 19:38:06.011573 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.011535 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:38:06.012291 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.012274 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j"] Sep 17 19:38:06.165567 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.165534 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/37e55478-59e1-49dd-9223-1d0d60abcaae-proxy-tls\") pod \"splitter-graph-fe2a8-59cd49fc6f-sz95j\" (UID: \"37e55478-59e1-49dd-9223-1d0d60abcaae\") " pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:06.165732 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.165578 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37e55478-59e1-49dd-9223-1d0d60abcaae-openshift-service-ca-bundle\") pod \"splitter-graph-fe2a8-59cd49fc6f-sz95j\" (UID: \"37e55478-59e1-49dd-9223-1d0d60abcaae\") " pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:06.205448 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.205402 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:38:06.266487 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.266456 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/37e55478-59e1-49dd-9223-1d0d60abcaae-proxy-tls\") pod \"splitter-graph-fe2a8-59cd49fc6f-sz95j\" (UID: \"37e55478-59e1-49dd-9223-1d0d60abcaae\") " pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:06.266647 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.266504 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37e55478-59e1-49dd-9223-1d0d60abcaae-openshift-service-ca-bundle\") pod \"splitter-graph-fe2a8-59cd49fc6f-sz95j\" (UID: \"37e55478-59e1-49dd-9223-1d0d60abcaae\") " pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:06.267219 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.267146 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37e55478-59e1-49dd-9223-1d0d60abcaae-openshift-service-ca-bundle\") pod \"splitter-graph-fe2a8-59cd49fc6f-sz95j\" (UID: \"37e55478-59e1-49dd-9223-1d0d60abcaae\") " pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:06.269046 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.269023 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/37e55478-59e1-49dd-9223-1d0d60abcaae-proxy-tls\") pod \"splitter-graph-fe2a8-59cd49fc6f-sz95j\" (UID: \"37e55478-59e1-49dd-9223-1d0d60abcaae\") " pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:06.313650 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.313617 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:06.588147 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:06.588121 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j"] Sep 17 19:38:06.590914 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:38:06.590872 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37e55478_59e1_49dd_9223_1d0d60abcaae.slice/crio-940b05c2bacbc461e7c3c8f26768c9fe5a84ab97b745a0f41236c4b73b13f7c5 WatchSource:0}: Error finding container 940b05c2bacbc461e7c3c8f26768c9fe5a84ab97b745a0f41236c4b73b13f7c5: Status 404 returned error can't find the container with id 940b05c2bacbc461e7c3c8f26768c9fe5a84ab97b745a0f41236c4b73b13f7c5 Sep 17 19:38:07.564216 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:07.564171 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" event={"ID":"37e55478-59e1-49dd-9223-1d0d60abcaae","Type":"ContainerStarted","Data":"fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae"} Sep 17 19:38:07.564693 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:07.564252 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" event={"ID":"37e55478-59e1-49dd-9223-1d0d60abcaae","Type":"ContainerStarted","Data":"940b05c2bacbc461e7c3c8f26768c9fe5a84ab97b745a0f41236c4b73b13f7c5"} Sep 17 19:38:07.564693 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:07.564287 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:07.582919 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:07.582849 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" podStartSLOduration=2.5828330189999997 podStartE2EDuration="2.582833019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:38:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:38:07.58178738 +0000 UTC m=+1891.535408750" watchObservedRunningTime="2026-09-17 19:38:07.582833019 +0000 UTC m=+1891.536454373" Sep 17 19:38:09.732493 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:09.732447 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" podUID="3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" containerName="sequence-graph-cedb0" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:38:09.733202 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:09.732553 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:38:13.572847 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:13.572820 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:14.732644 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:14.732603 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" podUID="3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" containerName="sequence-graph-cedb0" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:38:17.199780 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:17.199745 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j"] Sep 17 19:38:17.200193 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:17.199973 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" podUID="37e55478-59e1-49dd-9223-1d0d60abcaae" containerName="splitter-graph-fe2a8" containerID="cri-o://fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae" gracePeriod=30 Sep 17 19:38:17.246862 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:17.246823 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:38:17.336719 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:17.336673 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:38:17.431673 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:17.431640 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:38:17.501372 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:17.501294 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:38:17.646753 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:17.646720 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:38:17.752581 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:17.752495 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:38:18.572134 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:18.572093 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" podUID="37e55478-59e1-49dd-9223-1d0d60abcaae" containerName="splitter-graph-fe2a8" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:38:19.732829 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:19.732787 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" podUID="3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" containerName="sequence-graph-cedb0" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:38:23.571701 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:23.571654 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" podUID="37e55478-59e1-49dd-9223-1d0d60abcaae" containerName="splitter-graph-fe2a8" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:38:24.732812 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:24.732772 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" podUID="3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" containerName="sequence-graph-cedb0" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:38:27.106642 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.106617 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:38:27.121878 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.121858 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-proxy-tls\") pod \"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951\" (UID: \"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951\") " Sep 17 19:38:27.121974 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.121891 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-openshift-service-ca-bundle\") pod \"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951\" (UID: \"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951\") " Sep 17 19:38:27.122241 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.122218 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" (UID: "3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:38:27.123942 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.123922 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" (UID: "3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:38:27.222753 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.222675 2552 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-proxy-tls\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:38:27.222753 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.222702 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951-openshift-service-ca-bundle\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:38:27.634586 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.634554 2552 generic.go:356] "Generic (PLEG): container finished" podID="3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" containerID="1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8" exitCode=0 Sep 17 19:38:27.634743 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.634596 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" Sep 17 19:38:27.634743 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.634603 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" event={"ID":"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951","Type":"ContainerDied","Data":"1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8"} Sep 17 19:38:27.634743 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.634644 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l" event={"ID":"3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951","Type":"ContainerDied","Data":"d83071cb6532f729200e59105a682cb5361322b947a4080a1734c13a92832d93"} Sep 17 19:38:27.634743 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.634663 2552 scope.go:117] "RemoveContainer" containerID="1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8" Sep 17 19:38:27.643036 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.643018 2552 scope.go:117] "RemoveContainer" containerID="1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8" Sep 17 19:38:27.643306 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:38:27.643287 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8\": container with ID starting with 1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8 not found: ID does not exist" containerID="1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8" Sep 17 19:38:27.643350 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.643315 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8"} err="failed to get container status \"1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8\": rpc error: code = NotFound desc = could not find container \"1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8\": container with ID starting with 1b014e2e98ebc2df4e42aef75c8d363707f4d20eda6058bfec47065e3cf56ed8 not found: ID does not exist" Sep 17 19:38:27.654299 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.654276 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l"] Sep 17 19:38:27.658037 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:27.658018 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-cedb0-c565b857f-8kg7l"] Sep 17 19:38:28.514817 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:28.514781 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" path="/var/lib/kubelet/pods/3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951/volumes" Sep 17 19:38:28.572339 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:28.572301 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" podUID="37e55478-59e1-49dd-9223-1d0d60abcaae" containerName="splitter-graph-fe2a8" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:38:28.572514 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:28.572410 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:33.572166 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:33.572119 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" podUID="37e55478-59e1-49dd-9223-1d0d60abcaae" containerName="splitter-graph-fe2a8" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:38:38.571863 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:38.571825 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" podUID="37e55478-59e1-49dd-9223-1d0d60abcaae" containerName="splitter-graph-fe2a8" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:38:43.571851 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:43.571813 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" podUID="37e55478-59e1-49dd-9223-1d0d60abcaae" containerName="splitter-graph-fe2a8" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:38:47.385197 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.385175 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:47.469055 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.468972 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37e55478-59e1-49dd-9223-1d0d60abcaae-openshift-service-ca-bundle\") pod \"37e55478-59e1-49dd-9223-1d0d60abcaae\" (UID: \"37e55478-59e1-49dd-9223-1d0d60abcaae\") " Sep 17 19:38:47.469055 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.469042 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/37e55478-59e1-49dd-9223-1d0d60abcaae-proxy-tls\") pod \"37e55478-59e1-49dd-9223-1d0d60abcaae\" (UID: \"37e55478-59e1-49dd-9223-1d0d60abcaae\") " Sep 17 19:38:47.469378 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.469352 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37e55478-59e1-49dd-9223-1d0d60abcaae-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "37e55478-59e1-49dd-9223-1d0d60abcaae" (UID: "37e55478-59e1-49dd-9223-1d0d60abcaae"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:38:47.471241 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.471214 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37e55478-59e1-49dd-9223-1d0d60abcaae-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "37e55478-59e1-49dd-9223-1d0d60abcaae" (UID: "37e55478-59e1-49dd-9223-1d0d60abcaae"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:38:47.569871 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.569839 2552 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/37e55478-59e1-49dd-9223-1d0d60abcaae-proxy-tls\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:38:47.569871 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.569865 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/37e55478-59e1-49dd-9223-1d0d60abcaae-openshift-service-ca-bundle\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:38:47.706242 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.706205 2552 generic.go:356] "Generic (PLEG): container finished" podID="37e55478-59e1-49dd-9223-1d0d60abcaae" containerID="fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae" exitCode=0 Sep 17 19:38:47.706401 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.706252 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" Sep 17 19:38:47.706401 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.706300 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" event={"ID":"37e55478-59e1-49dd-9223-1d0d60abcaae","Type":"ContainerDied","Data":"fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae"} Sep 17 19:38:47.706401 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.706351 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j" event={"ID":"37e55478-59e1-49dd-9223-1d0d60abcaae","Type":"ContainerDied","Data":"940b05c2bacbc461e7c3c8f26768c9fe5a84ab97b745a0f41236c4b73b13f7c5"} Sep 17 19:38:47.706401 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.706373 2552 scope.go:117] "RemoveContainer" containerID="fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae" Sep 17 19:38:47.714789 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.714774 2552 scope.go:117] "RemoveContainer" containerID="fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae" Sep 17 19:38:47.715066 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:38:47.715043 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae\": container with ID starting with fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae not found: ID does not exist" containerID="fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae" Sep 17 19:38:47.715131 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.715074 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae"} err="failed to get container status \"fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae\": rpc error: code = NotFound desc = could not find container \"fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae\": container with ID starting with fd592df5511e5f5d28902025c008807d0ae570802052875fe1561dabaaaa97ae not found: ID does not exist" Sep 17 19:38:47.729374 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.729322 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j"] Sep 17 19:38:47.732925 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:47.732892 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-fe2a8-59cd49fc6f-sz95j"] Sep 17 19:38:48.515045 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:38:48.515017 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37e55478-59e1-49dd-9223-1d0d60abcaae" path="/var/lib/kubelet/pods/37e55478-59e1-49dd-9223-1d0d60abcaae/volumes" Sep 17 19:39:07.232833 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.232796 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh"] Sep 17 19:39:07.233376 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.233133 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37e55478-59e1-49dd-9223-1d0d60abcaae" containerName="splitter-graph-fe2a8" Sep 17 19:39:07.233376 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.233146 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="37e55478-59e1-49dd-9223-1d0d60abcaae" containerName="splitter-graph-fe2a8" Sep 17 19:39:07.233376 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.233155 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" containerName="sequence-graph-cedb0" Sep 17 19:39:07.233376 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.233160 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" containerName="sequence-graph-cedb0" Sep 17 19:39:07.233376 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.233207 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="37e55478-59e1-49dd-9223-1d0d60abcaae" containerName="splitter-graph-fe2a8" Sep 17 19:39:07.233376 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.233215 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c2b6c6e-b9de-4cd7-a0b5-e9bf3e922951" containerName="sequence-graph-cedb0" Sep 17 19:39:07.237558 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.237540 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:39:07.239303 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.239280 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:39:07.240397 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.240379 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-26209-serving-cert\"" Sep 17 19:39:07.240475 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.240398 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-26209-kube-rbac-proxy-sar-config\"" Sep 17 19:39:07.240475 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.240432 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 19:39:07.240580 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.240559 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tmb8q\"" Sep 17 19:39:07.245122 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.245100 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh"] Sep 17 19:39:07.313184 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.313140 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/315c59d5-a8e6-44fc-beda-df83a3269ae4-openshift-service-ca-bundle\") pod \"switch-graph-26209-76f8f79868-h8krh\" (UID: \"315c59d5-a8e6-44fc-beda-df83a3269ae4\") " pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:39:07.313354 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.313198 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/315c59d5-a8e6-44fc-beda-df83a3269ae4-proxy-tls\") pod \"switch-graph-26209-76f8f79868-h8krh\" (UID: \"315c59d5-a8e6-44fc-beda-df83a3269ae4\") " pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:39:07.414379 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.414345 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/315c59d5-a8e6-44fc-beda-df83a3269ae4-openshift-service-ca-bundle\") pod \"switch-graph-26209-76f8f79868-h8krh\" (UID: \"315c59d5-a8e6-44fc-beda-df83a3269ae4\") " pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:39:07.414379 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.414379 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/315c59d5-a8e6-44fc-beda-df83a3269ae4-proxy-tls\") pod \"switch-graph-26209-76f8f79868-h8krh\" (UID: \"315c59d5-a8e6-44fc-beda-df83a3269ae4\") " pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:39:07.414575 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:39:07.414522 2552 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-26209-serving-cert: secret "switch-graph-26209-serving-cert" not found Sep 17 19:39:07.414610 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:39:07.414576 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/315c59d5-a8e6-44fc-beda-df83a3269ae4-proxy-tls podName:315c59d5-a8e6-44fc-beda-df83a3269ae4 nodeName:}" failed. No retries permitted until 2026-09-17 19:39:07.914560466 +0000 UTC m=+1951.868181806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/315c59d5-a8e6-44fc-beda-df83a3269ae4-proxy-tls") pod "switch-graph-26209-76f8f79868-h8krh" (UID: "315c59d5-a8e6-44fc-beda-df83a3269ae4") : secret "switch-graph-26209-serving-cert" not found Sep 17 19:39:07.415148 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.415125 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/315c59d5-a8e6-44fc-beda-df83a3269ae4-openshift-service-ca-bundle\") pod \"switch-graph-26209-76f8f79868-h8krh\" (UID: \"315c59d5-a8e6-44fc-beda-df83a3269ae4\") " pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:39:07.918721 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:07.918680 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/315c59d5-a8e6-44fc-beda-df83a3269ae4-proxy-tls\") pod \"switch-graph-26209-76f8f79868-h8krh\" (UID: \"315c59d5-a8e6-44fc-beda-df83a3269ae4\") " pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:39:07.918938 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:39:07.918870 2552 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-26209-serving-cert: secret "switch-graph-26209-serving-cert" not found Sep 17 19:39:07.918996 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:39:07.918960 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/315c59d5-a8e6-44fc-beda-df83a3269ae4-proxy-tls podName:315c59d5-a8e6-44fc-beda-df83a3269ae4 nodeName:}" failed. No retries permitted until 2026-09-17 19:39:08.918939992 +0000 UTC m=+1952.872561335 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/315c59d5-a8e6-44fc-beda-df83a3269ae4-proxy-tls") pod "switch-graph-26209-76f8f79868-h8krh" (UID: "315c59d5-a8e6-44fc-beda-df83a3269ae4") : secret "switch-graph-26209-serving-cert" not found Sep 17 19:39:08.242263 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:08.242176 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:39:08.928032 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:08.927985 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/315c59d5-a8e6-44fc-beda-df83a3269ae4-proxy-tls\") pod \"switch-graph-26209-76f8f79868-h8krh\" (UID: \"315c59d5-a8e6-44fc-beda-df83a3269ae4\") " pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:39:08.930463 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:08.930442 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/315c59d5-a8e6-44fc-beda-df83a3269ae4-proxy-tls\") pod \"switch-graph-26209-76f8f79868-h8krh\" (UID: \"315c59d5-a8e6-44fc-beda-df83a3269ae4\") " pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:39:09.047715 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:09.047683 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:39:09.315165 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:09.315133 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh"] Sep 17 19:39:09.318028 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:39:09.317998 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod315c59d5_a8e6_44fc_beda_df83a3269ae4.slice/crio-90836c7b3dac9578c5f38f9b0ec9df35e81a663e4a5e805b6e00d741d4ce0081 WatchSource:0}: Error finding container 90836c7b3dac9578c5f38f9b0ec9df35e81a663e4a5e805b6e00d741d4ce0081: Status 404 returned error can't find the container with id 90836c7b3dac9578c5f38f9b0ec9df35e81a663e4a5e805b6e00d741d4ce0081 Sep 17 19:39:09.790364 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:09.790326 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" event={"ID":"315c59d5-a8e6-44fc-beda-df83a3269ae4","Type":"ContainerStarted","Data":"e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2"} Sep 17 19:39:09.790543 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:09.790387 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" event={"ID":"315c59d5-a8e6-44fc-beda-df83a3269ae4","Type":"ContainerStarted","Data":"90836c7b3dac9578c5f38f9b0ec9df35e81a663e4a5e805b6e00d741d4ce0081"} Sep 17 19:39:09.790543 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:09.790460 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:39:09.809290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:09.809235 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" podStartSLOduration=2.80921426 podStartE2EDuration="2.80921426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:39:09.808762452 +0000 UTC m=+1953.762383811" watchObservedRunningTime="2026-09-17 19:39:09.80921426 +0000 UTC m=+1953.762835618" Sep 17 19:39:15.798621 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:15.798589 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:39:27.428452 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:27.428416 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79"] Sep 17 19:39:27.431548 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:27.431531 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:39:27.434000 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:27.433976 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-21196-serving-cert\"" Sep 17 19:39:27.434139 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:27.434072 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-21196-kube-rbac-proxy-sar-config\"" Sep 17 19:39:27.434350 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:27.434327 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:39:27.440779 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:27.440746 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79"] Sep 17 19:39:27.571955 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:27.571893 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c78e9d00-bd33-4e9f-a25a-5deef211901d-openshift-service-ca-bundle\") pod \"splitter-graph-21196-646b77cb57-jxs79\" (UID: \"c78e9d00-bd33-4e9f-a25a-5deef211901d\") " pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:39:27.572123 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:27.571965 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c78e9d00-bd33-4e9f-a25a-5deef211901d-proxy-tls\") pod \"splitter-graph-21196-646b77cb57-jxs79\" (UID: \"c78e9d00-bd33-4e9f-a25a-5deef211901d\") " pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:39:27.673154 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:27.673113 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c78e9d00-bd33-4e9f-a25a-5deef211901d-openshift-service-ca-bundle\") pod \"splitter-graph-21196-646b77cb57-jxs79\" (UID: \"c78e9d00-bd33-4e9f-a25a-5deef211901d\") " pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:39:27.673336 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:27.673177 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c78e9d00-bd33-4e9f-a25a-5deef211901d-proxy-tls\") pod \"splitter-graph-21196-646b77cb57-jxs79\" (UID: \"c78e9d00-bd33-4e9f-a25a-5deef211901d\") " pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:39:27.673404 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:39:27.673342 2552 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-21196-serving-cert: secret "splitter-graph-21196-serving-cert" not found Sep 17 19:39:27.673456 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:39:27.673432 2552 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c78e9d00-bd33-4e9f-a25a-5deef211901d-proxy-tls podName:c78e9d00-bd33-4e9f-a25a-5deef211901d nodeName:}" failed. No retries permitted until 2026-09-17 19:39:28.173410197 +0000 UTC m=+1972.127031547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/c78e9d00-bd33-4e9f-a25a-5deef211901d-proxy-tls") pod "splitter-graph-21196-646b77cb57-jxs79" (UID: "c78e9d00-bd33-4e9f-a25a-5deef211901d") : secret "splitter-graph-21196-serving-cert" not found Sep 17 19:39:27.673753 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:27.673734 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c78e9d00-bd33-4e9f-a25a-5deef211901d-openshift-service-ca-bundle\") pod \"splitter-graph-21196-646b77cb57-jxs79\" (UID: \"c78e9d00-bd33-4e9f-a25a-5deef211901d\") " pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:39:27.747614 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:27.747528 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:39:28.177402 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:28.177362 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c78e9d00-bd33-4e9f-a25a-5deef211901d-proxy-tls\") pod \"splitter-graph-21196-646b77cb57-jxs79\" (UID: \"c78e9d00-bd33-4e9f-a25a-5deef211901d\") " pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:39:28.179867 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:28.179846 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c78e9d00-bd33-4e9f-a25a-5deef211901d-proxy-tls\") pod \"splitter-graph-21196-646b77cb57-jxs79\" (UID: \"c78e9d00-bd33-4e9f-a25a-5deef211901d\") " pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:39:28.341659 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:28.341621 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:39:28.611269 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:28.611238 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79"] Sep 17 19:39:28.613981 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:39:28.613948 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc78e9d00_bd33_4e9f_a25a_5deef211901d.slice/crio-d2179996d5ecbd9a663bd2578347d3d4303813f4348f25f017d99dcfa0d154fc WatchSource:0}: Error finding container d2179996d5ecbd9a663bd2578347d3d4303813f4348f25f017d99dcfa0d154fc: Status 404 returned error can't find the container with id d2179996d5ecbd9a663bd2578347d3d4303813f4348f25f017d99dcfa0d154fc Sep 17 19:39:28.859611 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:28.859574 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" event={"ID":"c78e9d00-bd33-4e9f-a25a-5deef211901d","Type":"ContainerStarted","Data":"c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1"} Sep 17 19:39:28.859793 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:28.859631 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" event={"ID":"c78e9d00-bd33-4e9f-a25a-5deef211901d","Type":"ContainerStarted","Data":"d2179996d5ecbd9a663bd2578347d3d4303813f4348f25f017d99dcfa0d154fc"} Sep 17 19:39:28.859793 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:28.859668 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:39:28.881369 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:28.878133 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" podStartSLOduration=1.878114397 podStartE2EDuration="1.878114397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 19:39:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:39:28.875667992 +0000 UTC m=+1972.829289349" watchObservedRunningTime="2026-09-17 19:39:28.878114397 +0000 UTC m=+1972.831735756" Sep 17 19:39:34.868685 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:34.868655 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:39:51.663853 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:39:51.663821 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:40:31.670397 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:40:31.670362 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:40:57.309701 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:40:57.309669 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:45:59.629917 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:45:59.629855 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:47:42.580704 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:47:42.580665 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79"] Sep 17 19:47:42.581237 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:47:42.580860 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" podUID="c78e9d00-bd33-4e9f-a25a-5deef211901d" containerName="splitter-graph-21196" containerID="cri-o://c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1" gracePeriod=30 Sep 17 19:47:42.626301 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:47:42.626272 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:47:42.718553 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:47:42.718496 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:47:42.778177 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:47:42.778140 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:47:44.867193 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:47:44.867159 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" podUID="c78e9d00-bd33-4e9f-a25a-5deef211901d" containerName="splitter-graph-21196" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:47:49.866857 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:47:49.866811 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" podUID="c78e9d00-bd33-4e9f-a25a-5deef211901d" containerName="splitter-graph-21196" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:47:54.867013 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:47:54.866977 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" podUID="c78e9d00-bd33-4e9f-a25a-5deef211901d" containerName="splitter-graph-21196" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:47:54.867368 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:47:54.867073 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:47:59.868166 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:47:59.868128 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" podUID="c78e9d00-bd33-4e9f-a25a-5deef211901d" containerName="splitter-graph-21196" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:48:04.867618 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:04.867583 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" podUID="c78e9d00-bd33-4e9f-a25a-5deef211901d" containerName="splitter-graph-21196" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:48:09.867644 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:09.867607 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" podUID="c78e9d00-bd33-4e9f-a25a-5deef211901d" containerName="splitter-graph-21196" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:48:12.716122 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.716093 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:48:12.723698 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.723674 2552 generic.go:356] "Generic (PLEG): container finished" podID="c78e9d00-bd33-4e9f-a25a-5deef211901d" containerID="c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1" exitCode=0 Sep 17 19:48:12.723821 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.723714 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" Sep 17 19:48:12.723821 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.723748 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" event={"ID":"c78e9d00-bd33-4e9f-a25a-5deef211901d","Type":"ContainerDied","Data":"c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1"} Sep 17 19:48:12.723821 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.723793 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79" event={"ID":"c78e9d00-bd33-4e9f-a25a-5deef211901d","Type":"ContainerDied","Data":"d2179996d5ecbd9a663bd2578347d3d4303813f4348f25f017d99dcfa0d154fc"} Sep 17 19:48:12.723821 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.723815 2552 scope.go:117] "RemoveContainer" containerID="c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1" Sep 17 19:48:12.731450 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.731435 2552 scope.go:117] "RemoveContainer" containerID="c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1" Sep 17 19:48:12.731698 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:48:12.731682 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1\": container with ID starting with c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1 not found: ID does not exist" containerID="c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1" Sep 17 19:48:12.731742 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.731708 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1"} err="failed to get container status \"c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1\": rpc error: code = NotFound desc = could not find container \"c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1\": container with ID starting with c4494534b00bccae5f768632d58e6d15e71b4adfd60e13babe7e67b1594b92a1 not found: ID does not exist" Sep 17 19:48:12.783577 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.783548 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c78e9d00-bd33-4e9f-a25a-5deef211901d-openshift-service-ca-bundle\") pod \"c78e9d00-bd33-4e9f-a25a-5deef211901d\" (UID: \"c78e9d00-bd33-4e9f-a25a-5deef211901d\") " Sep 17 19:48:12.783771 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.783624 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c78e9d00-bd33-4e9f-a25a-5deef211901d-proxy-tls\") pod \"c78e9d00-bd33-4e9f-a25a-5deef211901d\" (UID: \"c78e9d00-bd33-4e9f-a25a-5deef211901d\") " Sep 17 19:48:12.783893 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.783870 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c78e9d00-bd33-4e9f-a25a-5deef211901d-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "c78e9d00-bd33-4e9f-a25a-5deef211901d" (UID: "c78e9d00-bd33-4e9f-a25a-5deef211901d"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:48:12.785767 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.785748 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c78e9d00-bd33-4e9f-a25a-5deef211901d-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c78e9d00-bd33-4e9f-a25a-5deef211901d" (UID: "c78e9d00-bd33-4e9f-a25a-5deef211901d"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:48:12.884693 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.884592 2552 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c78e9d00-bd33-4e9f-a25a-5deef211901d-proxy-tls\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:48:12.884693 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:12.884639 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c78e9d00-bd33-4e9f-a25a-5deef211901d-openshift-service-ca-bundle\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:48:13.044620 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:13.044594 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79"] Sep 17 19:48:13.048557 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:13.048533 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-21196-646b77cb57-jxs79"] Sep 17 19:48:14.514643 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:48:14.514613 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c78e9d00-bd33-4e9f-a25a-5deef211901d" path="/var/lib/kubelet/pods/c78e9d00-bd33-4e9f-a25a-5deef211901d/volumes" Sep 17 19:49:50.986511 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:49:50.986478 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:54:37.890663 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:54:37.890594 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:55:27.345516 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:27.345478 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh"] Sep 17 19:55:27.346123 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:27.345706 2552 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" podUID="315c59d5-a8e6-44fc-beda-df83a3269ae4" containerName="switch-graph-26209" containerID="cri-o://e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2" gracePeriod=30 Sep 17 19:55:27.369859 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:27.369825 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:55:27.497611 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:27.497583 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:55:27.558277 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:27.558247 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:55:28.338574 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:28.338535 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:55:28.428579 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:28.428551 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:55:30.797027 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:30.796988 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" podUID="315c59d5-a8e6-44fc-beda-df83a3269ae4" containerName="switch-graph-26209" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:55:35.796960 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:35.796918 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" podUID="315c59d5-a8e6-44fc-beda-df83a3269ae4" containerName="switch-graph-26209" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:55:40.797590 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:40.797554 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" podUID="315c59d5-a8e6-44fc-beda-df83a3269ae4" containerName="switch-graph-26209" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:55:40.797998 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:40.797646 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:55:41.601641 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:41.601607 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:55:43.788330 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:43.788295 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:44.640150 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:44.640120 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:45.541911 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:45.541878 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:45.797448 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:45.797365 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" podUID="315c59d5-a8e6-44fc-beda-df83a3269ae4" containerName="switch-graph-26209" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:55:46.312730 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:46.312704 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:47.072372 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:47.072343 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:47.873362 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:47.873338 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:48.657768 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:48.657736 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:49.463761 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:49.463730 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:50.265125 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:50.265096 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:50.797232 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:50.797198 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" podUID="315c59d5-a8e6-44fc-beda-df83a3269ae4" containerName="switch-graph-26209" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:55:51.046150 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:51.046118 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:51.799381 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:51.799349 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:52.553269 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:52.553240 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:53.332087 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:53.332059 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:54.115579 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:54.115547 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:54.913564 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:54.913535 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-26209-76f8f79868-h8krh_315c59d5-a8e6-44fc-beda-df83a3269ae4/switch-graph-26209/0.log" Sep 17 19:55:55.797631 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:55.797589 2552 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" podUID="315c59d5-a8e6-44fc-beda-df83a3269ae4" containerName="switch-graph-26209" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 19:55:57.487690 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:57.487667 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:55:57.559949 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:57.559917 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/315c59d5-a8e6-44fc-beda-df83a3269ae4-proxy-tls\") pod \"315c59d5-a8e6-44fc-beda-df83a3269ae4\" (UID: \"315c59d5-a8e6-44fc-beda-df83a3269ae4\") " Sep 17 19:55:57.560096 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:57.559961 2552 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/315c59d5-a8e6-44fc-beda-df83a3269ae4-openshift-service-ca-bundle\") pod \"315c59d5-a8e6-44fc-beda-df83a3269ae4\" (UID: \"315c59d5-a8e6-44fc-beda-df83a3269ae4\") " Sep 17 19:55:57.560251 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:57.560229 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/315c59d5-a8e6-44fc-beda-df83a3269ae4-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "315c59d5-a8e6-44fc-beda-df83a3269ae4" (UID: "315c59d5-a8e6-44fc-beda-df83a3269ae4"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 19:55:57.561920 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:57.561888 2552 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/315c59d5-a8e6-44fc-beda-df83a3269ae4-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "315c59d5-a8e6-44fc-beda-df83a3269ae4" (UID: "315c59d5-a8e6-44fc-beda-df83a3269ae4"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 19:55:57.660555 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:57.660487 2552 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/315c59d5-a8e6-44fc-beda-df83a3269ae4-openshift-service-ca-bundle\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:55:57.660555 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:57.660510 2552 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/315c59d5-a8e6-44fc-beda-df83a3269ae4-proxy-tls\") on node \"ip-10-0-143-217.ec2.internal\" DevicePath \"\"" Sep 17 19:55:58.350363 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:58.350329 2552 generic.go:356] "Generic (PLEG): container finished" podID="315c59d5-a8e6-44fc-beda-df83a3269ae4" containerID="e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2" exitCode=0 Sep 17 19:55:58.350363 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:58.350374 2552 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" Sep 17 19:55:58.350657 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:58.350408 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" event={"ID":"315c59d5-a8e6-44fc-beda-df83a3269ae4","Type":"ContainerDied","Data":"e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2"} Sep 17 19:55:58.350657 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:58.350472 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh" event={"ID":"315c59d5-a8e6-44fc-beda-df83a3269ae4","Type":"ContainerDied","Data":"90836c7b3dac9578c5f38f9b0ec9df35e81a663e4a5e805b6e00d741d4ce0081"} Sep 17 19:55:58.350657 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:58.350495 2552 scope.go:117] "RemoveContainer" containerID="e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2" Sep 17 19:55:58.359935 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:58.359894 2552 scope.go:117] "RemoveContainer" containerID="e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2" Sep 17 19:55:58.360251 ip-10-0-143-217 kubenswrapper[2552]: E0917 19:55:58.360231 2552 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2\": container with ID starting with e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2 not found: ID does not exist" containerID="e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2" Sep 17 19:55:58.360339 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:58.360260 2552 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2"} err="failed to get container status \"e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2\": rpc error: code = NotFound desc = could not find container \"e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2\": container with ID starting with e69003d5739ff279a330e623db1468d06bb7fd670a6fc536ff76fbfb77ad96e2 not found: ID does not exist" Sep 17 19:55:58.372880 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:58.372860 2552 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh"] Sep 17 19:55:58.376783 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:58.376760 2552 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-26209-76f8f79868-h8krh"] Sep 17 19:55:58.515755 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:58.515725 2552 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="315c59d5-a8e6-44fc-beda-df83a3269ae4" path="/var/lib/kubelet/pods/315c59d5-a8e6-44fc-beda-df83a3269ae4/volumes" Sep 17 19:55:59.873481 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:55:59.873445 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:56:00.024471 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:00.024432 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:56:00.125992 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:00.125892 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:56:02.998107 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:02.998075 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ls7s4_c5ac3838-c267-4741-8925-8c2e90745194/global-pull-secret-syncer/0.log" Sep 17 19:56:03.131860 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:03.131825 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" Sep 17 19:56:03.268190 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:03.268165 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-217.ec2.internal_9f44ab85f08a059c30913c6fa3110b60/haproxy/0.log" Sep 17 19:56:05.772257 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:05.772223 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:56:05.827811 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:05.827783 2552 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5w586_d78749d7-d120-4da2-a5fd-29d6bbe672c3/konnectivity-agent/0.log" limit=1024 Sep 17 19:56:07.495007 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:07.494976 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5984459ff8-9bh24_96190262-af6d-4736-a306-9f1dc4014513/monitoring-plugin/0.log" Sep 17 19:56:07.645982 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:07.645954 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mjckh_568754ba-19bf-4224-b0b0-2bfbfbee6011/node-exporter/0.log" Sep 17 19:56:07.680569 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:07.680541 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mjckh_568754ba-19bf-4224-b0b0-2bfbfbee6011/kube-rbac-proxy/0.log" Sep 17 19:56:07.708492 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:07.708463 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mjckh_568754ba-19bf-4224-b0b0-2bfbfbee6011/init-textfile/0.log" Sep 17 19:56:09.809745 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:09.809719 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-ddkjb_9a309853-4490-437d-a9c7-93136e1acbf8/networking-console-plugin/0.log" Sep 17 19:56:11.764404 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.764371 2552 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq"] Sep 17 19:56:11.764748 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.764659 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c78e9d00-bd33-4e9f-a25a-5deef211901d" containerName="splitter-graph-21196" Sep 17 19:56:11.764748 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.764670 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="c78e9d00-bd33-4e9f-a25a-5deef211901d" containerName="splitter-graph-21196" Sep 17 19:56:11.764748 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.764684 2552 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="315c59d5-a8e6-44fc-beda-df83a3269ae4" containerName="switch-graph-26209" Sep 17 19:56:11.764748 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.764689 2552 state_mem.go:107] "Deleted CPUSet assignment" podUID="315c59d5-a8e6-44fc-beda-df83a3269ae4" containerName="switch-graph-26209" Sep 17 19:56:11.764748 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.764733 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="c78e9d00-bd33-4e9f-a25a-5deef211901d" containerName="splitter-graph-21196" Sep 17 19:56:11.764748 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.764743 2552 memory_manager.go:368] "RemoveStaleState removing state" podUID="315c59d5-a8e6-44fc-beda-df83a3269ae4" containerName="switch-graph-26209" Sep 17 19:56:11.769013 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.768994 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.771508 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.771485 2552 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9dgbs\"/\"default-dockercfg-lfrpr\"" Sep 17 19:56:11.771608 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.771579 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9dgbs\"/\"openshift-service-ca.crt\"" Sep 17 19:56:11.772591 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.772575 2552 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9dgbs\"/\"kube-root-ca.crt\"" Sep 17 19:56:11.776259 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.776237 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq"] Sep 17 19:56:11.864002 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.863965 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/10adbdcb-2159-4793-8e3b-7e2cd5140d55-lib-modules\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.864002 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.864004 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/10adbdcb-2159-4793-8e3b-7e2cd5140d55-proc\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.864226 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.864025 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/10adbdcb-2159-4793-8e3b-7e2cd5140d55-sys\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.864226 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.864083 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/10adbdcb-2159-4793-8e3b-7e2cd5140d55-podres\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.864226 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.864126 2552 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzw42\" (UniqueName: \"kubernetes.io/projected/10adbdcb-2159-4793-8e3b-7e2cd5140d55-kube-api-access-gzw42\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.965214 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.965177 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/10adbdcb-2159-4793-8e3b-7e2cd5140d55-lib-modules\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.965214 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.965217 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/10adbdcb-2159-4793-8e3b-7e2cd5140d55-proc\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.965424 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.965248 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/10adbdcb-2159-4793-8e3b-7e2cd5140d55-sys\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.965424 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.965273 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/10adbdcb-2159-4793-8e3b-7e2cd5140d55-podres\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.965424 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.965305 2552 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzw42\" (UniqueName: \"kubernetes.io/projected/10adbdcb-2159-4793-8e3b-7e2cd5140d55-kube-api-access-gzw42\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.965424 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.965314 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/10adbdcb-2159-4793-8e3b-7e2cd5140d55-proc\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.965424 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.965316 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/10adbdcb-2159-4793-8e3b-7e2cd5140d55-sys\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.965424 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.965361 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/10adbdcb-2159-4793-8e3b-7e2cd5140d55-lib-modules\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.965424 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.965377 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/10adbdcb-2159-4793-8e3b-7e2cd5140d55-podres\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:11.973627 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:11.973596 2552 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzw42\" (UniqueName: \"kubernetes.io/projected/10adbdcb-2159-4793-8e3b-7e2cd5140d55-kube-api-access-gzw42\") pod \"perf-node-gather-daemonset-cj6zq\" (UID: \"10adbdcb-2159-4793-8e3b-7e2cd5140d55\") " pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:12.079161 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:12.079082 2552 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:12.346102 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:12.345964 2552 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq"] Sep 17 19:56:12.348580 ip-10-0-143-217 kubenswrapper[2552]: W0917 19:56:12.348551 2552 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod10adbdcb_2159_4793_8e3b_7e2cd5140d55.slice/crio-ba135f493825a62797998db00d00d9bb05ce184cacb4ce35125fc8d5d5cfa27c WatchSource:0}: Error finding container ba135f493825a62797998db00d00d9bb05ce184cacb4ce35125fc8d5d5cfa27c: Status 404 returned error can't find the container with id ba135f493825a62797998db00d00d9bb05ce184cacb4ce35125fc8d5d5cfa27c Sep 17 19:56:12.398712 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:12.398677 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" event={"ID":"10adbdcb-2159-4793-8e3b-7e2cd5140d55","Type":"ContainerStarted","Data":"ba135f493825a62797998db00d00d9bb05ce184cacb4ce35125fc8d5d5cfa27c"} Sep 17 19:56:12.403432 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:12.403409 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5bxxw_bce04825-89fe-47fa-a72f-0a24aa333a75/dns/0.log" Sep 17 19:56:12.432648 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:12.432624 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5bxxw_bce04825-89fe-47fa-a72f-0a24aa333a75/kube-rbac-proxy/0.log" Sep 17 19:56:12.612237 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:12.612148 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ndhbz_ac559d16-e42b-4873-bd8e-abda5c4137fd/dns-node-resolver/0.log" Sep 17 19:56:13.138049 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:13.138025 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lsmrz_4bcaefd0-fa6b-4bf6-aa9b-eee050c3626a/node-ca/0.log" Sep 17 19:56:13.406462 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:13.406379 2552 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" event={"ID":"10adbdcb-2159-4793-8e3b-7e2cd5140d55","Type":"ContainerStarted","Data":"6a3149fd388e1ae9d26e7b05114e371a3970093505a06204ae6d73b7752fb87b"} Sep 17 19:56:13.406603 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:13.406493 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:13.422911 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:13.422864 2552 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" podStartSLOduration=2.422850643 podStartE2EDuration="2.422850643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 19:56:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 19:56:13.422327448 +0000 UTC m=+2977.375948817" watchObservedRunningTime="2026-09-17 19:56:13.422850643 +0000 UTC m=+2977.376472000" Sep 17 19:56:14.336521 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:14.336489 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-95lc8_36e00d1e-728d-4377-bca4-5f14d2a88b7e/serve-healthcheck-canary/0.log" Sep 17 19:56:14.901026 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:14.901000 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hlh59_5404dd23-6e97-439f-a95d-8b3c8e533de8/kube-rbac-proxy/0.log" Sep 17 19:56:14.923290 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:14.923259 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hlh59_5404dd23-6e97-439f-a95d-8b3c8e533de8/exporter/0.log" Sep 17 19:56:14.945602 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:14.945571 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hlh59_5404dd23-6e97-439f-a95d-8b3c8e533de8/extractor/0.log" Sep 17 19:56:17.379847 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:17.379820 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-66576f5db6-wmgkk_635348b2-c621-4fb7-a1f4-955986d0897f/manager/0.log" Sep 17 19:56:17.436779 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:17.436749 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-7b954ffc95-dbzz9_71c602b9-85bc-4f7a-8384-d9c647cd6e56/server/0.log" Sep 17 19:56:17.528547 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:17.528518 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7b966d7984-cfmsv_cdfa09dc-3717-4dba-99a4-a3b97484e3b8/manager/0.log" Sep 17 19:56:19.421078 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:19.421051 2552 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9dgbs/perf-node-gather-daemonset-cj6zq" Sep 17 19:56:22.447885 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:22.447833 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-zsjnd_33012019-3ef9-475a-a6ab-f7e445f79ea9/kube-storage-version-migrator-operator/1.log" Sep 17 19:56:22.448947 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:22.448927 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-zsjnd_33012019-3ef9-475a-a6ab-f7e445f79ea9/kube-storage-version-migrator-operator/0.log" Sep 17 19:56:23.498034 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:23.497960 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4nvkk_56c11fe2-25e7-4bb2-b789-ea4178f73d25/kube-multus/0.log" Sep 17 19:56:23.520435 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:23.520409 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8zwvj_e96469d5-497d-4b64-ad08-465aa483e44a/kube-multus-additional-cni-plugins/0.log" Sep 17 19:56:23.542648 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:23.542614 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8zwvj_e96469d5-497d-4b64-ad08-465aa483e44a/egress-router-binary-copy/0.log" Sep 17 19:56:23.564736 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:23.564711 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8zwvj_e96469d5-497d-4b64-ad08-465aa483e44a/cni-plugins/0.log" Sep 17 19:56:23.585868 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:23.585842 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8zwvj_e96469d5-497d-4b64-ad08-465aa483e44a/bond-cni-plugin/0.log" Sep 17 19:56:23.609511 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:23.609488 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8zwvj_e96469d5-497d-4b64-ad08-465aa483e44a/routeoverride-cni/0.log" Sep 17 19:56:23.632062 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:23.632043 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8zwvj_e96469d5-497d-4b64-ad08-465aa483e44a/whereabouts-cni-bincopy/0.log" Sep 17 19:56:23.653750 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:23.653732 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8zwvj_e96469d5-497d-4b64-ad08-465aa483e44a/whereabouts-cni/0.log" Sep 17 19:56:24.078607 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:24.078581 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-422mp_366e0b69-56de-45e9-9a8b-87f0a4c8b4bf/network-metrics-daemon/0.log" Sep 17 19:56:24.101324 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:24.101299 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-422mp_366e0b69-56de-45e9-9a8b-87f0a4c8b4bf/kube-rbac-proxy/0.log" Sep 17 19:56:25.171715 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:25.171682 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jt5kn_c439c32d-c42d-4795-8e5e-d4dcf5479e32/ovn-controller/0.log" Sep 17 19:56:25.192067 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:25.192044 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jt5kn_c439c32d-c42d-4795-8e5e-d4dcf5479e32/ovn-acl-logging/0.log" Sep 17 19:56:25.217945 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:25.217922 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jt5kn_c439c32d-c42d-4795-8e5e-d4dcf5479e32/kube-rbac-proxy-node/0.log" Sep 17 19:56:25.240830 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:25.240808 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jt5kn_c439c32d-c42d-4795-8e5e-d4dcf5479e32/kube-rbac-proxy-ovn-metrics/0.log" Sep 17 19:56:25.261843 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:25.261825 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jt5kn_c439c32d-c42d-4795-8e5e-d4dcf5479e32/northd/0.log" Sep 17 19:56:25.283312 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:25.283289 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jt5kn_c439c32d-c42d-4795-8e5e-d4dcf5479e32/nbdb/0.log" Sep 17 19:56:25.307922 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:25.307888 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jt5kn_c439c32d-c42d-4795-8e5e-d4dcf5479e32/sbdb/0.log" Sep 17 19:56:25.346819 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:25.346792 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jt5kn_c439c32d-c42d-4795-8e5e-d4dcf5479e32/ovnkube-controller/0.log" Sep 17 19:56:26.592368 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:26.592311 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-t7jj9_45c9424d-9eba-4232-9e83-85dff9e0132f/network-check-target-container/0.log" Sep 17 19:56:27.679307 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:27.679283 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-bnf7d_441acc97-89e4-47a7-9ea1-923c8e7e99a7/iptables-alerter/0.log" Sep 17 19:56:28.446671 ip-10-0-143-217 kubenswrapper[2552]: I0917 19:56:28.446642 2552 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-psqk9_27709855-de56-4854-a822-e6044d98d1a4/tuned/0.log"