Sep 17 15:53:34.415500 ip-10-0-128-222 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 17 15:53:34.415511 ip-10-0-128-222 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 17 15:53:34.415520 ip-10-0-128-222 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 17 15:53:34.415697 ip-10-0-128-222 systemd[1]: Failed to start Kubernetes Kubelet. Sep 17 15:53:44.664508 ip-10-0-128-222 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 17 15:53:44.664525 ip-10-0-128-222 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e454a362e7104f5686b8f42340079b52 -- Sep 17 15:55:41.386673 ip-10-0-128-222 systemd[1]: Starting Kubernetes Kubelet... Sep 17 15:55:41.748428 ip-10-0-128-222 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 15:55:41.748428 ip-10-0-128-222 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 17 15:55:41.748428 ip-10-0-128-222 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 15:55:41.748428 ip-10-0-128-222 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 17 15:55:41.749015 ip-10-0-128-222 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 17 15:55:41.749338 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.749241 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 17 15:55:41.753568 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753553 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 15:55:41.753568 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753569 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753573 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753576 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753580 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753583 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753586 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753588 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753591 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753594 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753596 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753599 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753601 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753605 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753607 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753610 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753613 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753616 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753620 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753624 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 15:55:41.753639 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753627 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753630 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753633 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753636 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753639 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753642 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753645 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753648 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753651 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753654 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753657 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753659 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753662 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753665 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753667 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753670 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753673 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753675 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753678 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753680 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 15:55:41.754171 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753683 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753685 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753688 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753690 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753692 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753695 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753697 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753700 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753703 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753705 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753708 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753710 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753713 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753715 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753718 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753721 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753723 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753726 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753729 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753731 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 15:55:41.754894 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753735 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753737 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753740 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753742 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753745 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753748 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753750 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753753 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753756 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753758 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753760 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753763 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753766 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753768 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753772 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753775 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753779 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753781 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753785 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753787 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 15:55:41.755417 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753790 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753792 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753795 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753798 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753801 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753810 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753814 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753817 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753820 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753822 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753825 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753828 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753831 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753833 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753836 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753839 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753841 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753843 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 15:55:41.755905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.753846 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 15:55:41.756535 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756519 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 15:55:41.756535 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756534 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756538 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756542 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756545 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756548 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756551 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756554 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756557 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756560 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756563 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756565 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756568 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756572 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756576 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756579 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756582 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756586 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756589 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756592 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 15:55:41.756595 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756595 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756598 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756601 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756604 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756608 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756611 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756614 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756617 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756620 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756623 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756625 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756628 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756631 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756633 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756636 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756639 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756642 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756644 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756647 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756649 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 15:55:41.757067 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756652 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756654 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756657 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756659 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756662 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756664 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756667 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756669 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756672 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756674 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756678 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756680 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756683 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756685 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756689 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756691 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756694 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756697 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756700 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756703 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 15:55:41.757624 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756706 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756709 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756711 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756714 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756717 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756719 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756722 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756724 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756727 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756730 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756732 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756736 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756738 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756741 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756744 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756746 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756749 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756753 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756757 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 15:55:41.758107 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756762 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756764 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756767 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756770 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756773 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756776 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756778 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756782 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756785 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756788 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756790 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756793 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756796 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756798 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756801 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756804 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756806 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756809 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756812 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 15:55:41.758583 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.756815 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757161 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757167 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757170 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757173 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757176 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757179 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757181 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757184 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757186 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757191 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757194 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757197 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757200 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757203 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757206 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757209 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757212 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757214 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 15:55:41.759031 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757217 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757220 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757223 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757226 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757228 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757231 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757233 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757236 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757239 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757241 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757243 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757246 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757248 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757251 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757253 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757256 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757258 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757262 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757264 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757267 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 15:55:41.759512 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757285 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757288 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757292 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757295 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757298 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757301 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757304 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757307 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757309 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757313 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757315 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757318 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757321 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757324 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757326 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757329 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757332 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757334 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757337 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757339 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 15:55:41.759985 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757342 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757344 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757347 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757350 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757352 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757354 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757357 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757360 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757362 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757365 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757367 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757370 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757373 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757375 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757378 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757380 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757383 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757385 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757388 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757390 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 15:55:41.760484 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757405 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757408 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757412 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757415 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757419 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757423 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757426 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757429 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757432 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757436 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757438 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757441 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757444 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757448 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757451 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757454 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757456 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757459 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757461 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 15:55:41.760977 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757464 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.757466 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757533 2580 flags.go:64] FLAG: --address="0.0.0.0" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757568 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757573 2580 flags.go:64] FLAG: --anonymous-auth="true" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757577 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757581 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757585 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757588 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757592 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757596 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757599 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757602 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757605 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757608 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757611 2580 flags.go:64] FLAG: --cgroup-root="" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757614 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757617 2580 flags.go:64] FLAG: --client-ca-file="" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757622 2580 flags.go:64] FLAG: --cloud-provider="external" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757625 2580 flags.go:64] FLAG: --cluster-dns="[]" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757630 2580 flags.go:64] FLAG: --cluster-domain="" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757633 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757636 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757639 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 17 15:55:41.761430 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757642 2580 flags.go:64] FLAG: --container-log-max-files="5" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757646 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757649 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757652 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757656 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757660 2580 flags.go:64] FLAG: --contention-profiling="false" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757662 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757665 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757668 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757671 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757680 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757683 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757686 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757689 2580 flags.go:64] FLAG: --enable-load-reader="false" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757691 2580 flags.go:64] FLAG: --enable-server="true" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757695 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757699 2580 flags.go:64] FLAG: --event-burst="100" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757702 2580 flags.go:64] FLAG: --event-qps="50" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757705 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757708 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757711 2580 flags.go:64] FLAG: --eviction-hard="" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757714 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757717 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757720 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757723 2580 flags.go:64] FLAG: --eviction-soft="" Sep 17 15:55:41.762004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757726 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757729 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757732 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757735 2580 flags.go:64] FLAG: --experimental-mounter-path="" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757738 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757741 2580 flags.go:64] FLAG: --fail-swap-on="true" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757744 2580 flags.go:64] FLAG: --feature-gates="" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757747 2580 flags.go:64] FLAG: --file-check-frequency="20s" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757752 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757756 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757759 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757762 2580 flags.go:64] FLAG: --healthz-port="10248" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757765 2580 flags.go:64] FLAG: --help="false" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757768 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-128-222.ec2.internal" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757771 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757774 2580 flags.go:64] FLAG: --http-check-frequency="20s" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757777 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757780 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757784 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757787 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757790 2580 flags.go:64] FLAG: --image-service-endpoint="" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757792 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757795 2580 flags.go:64] FLAG: --kube-api-burst="100" Sep 17 15:55:41.762674 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757798 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757801 2580 flags.go:64] FLAG: --kube-api-qps="50" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757804 2580 flags.go:64] FLAG: --kube-reserved="" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757807 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757810 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757813 2580 flags.go:64] FLAG: --kubelet-cgroups="" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757815 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757818 2580 flags.go:64] FLAG: --lock-file="" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757821 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757824 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757827 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757833 2580 flags.go:64] FLAG: --log-json-split-stream="false" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757836 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757839 2580 flags.go:64] FLAG: --log-text-split-stream="false" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757842 2580 flags.go:64] FLAG: --logging-format="text" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757845 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757848 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757852 2580 flags.go:64] FLAG: --manifest-url="" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757855 2580 flags.go:64] FLAG: --manifest-url-header="" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757859 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757862 2580 flags.go:64] FLAG: --max-open-files="1000000" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757866 2580 flags.go:64] FLAG: --max-pods="110" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757869 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757872 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757875 2580 flags.go:64] FLAG: --memory-manager-policy="None" Sep 17 15:55:41.763243 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757878 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757881 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757884 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757887 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757894 2580 flags.go:64] FLAG: --node-status-max-images="50" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757897 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757900 2580 flags.go:64] FLAG: --oom-score-adj="-999" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757903 2580 flags.go:64] FLAG: --pod-cidr="" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757906 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757911 2580 flags.go:64] FLAG: --pod-manifest-path="" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757914 2580 flags.go:64] FLAG: --pod-max-pids="-1" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757917 2580 flags.go:64] FLAG: --pods-per-core="0" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757920 2580 flags.go:64] FLAG: --port="10250" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757923 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757926 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bcb88d065edd4161" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757929 2580 flags.go:64] FLAG: --qos-reserved="" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757933 2580 flags.go:64] FLAG: --read-only-port="10255" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757935 2580 flags.go:64] FLAG: --register-node="true" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757938 2580 flags.go:64] FLAG: --register-with-taints="" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757942 2580 flags.go:64] FLAG: --registry-burst="10" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757945 2580 flags.go:64] FLAG: --registry-qps="5" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757948 2580 flags.go:64] FLAG: --reserved-cpus="" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757951 2580 flags.go:64] FLAG: --reserved-memory="" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757954 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757958 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 17 15:55:41.763857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757961 2580 flags.go:64] FLAG: --rotate-certificates="false" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757964 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757967 2580 flags.go:64] FLAG: --runonce="false" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757970 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757972 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757975 2580 flags.go:64] FLAG: --seccomp-default="false" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757978 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757981 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757984 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757987 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757990 2580 flags.go:64] FLAG: --storage-driver-password="root" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757992 2580 flags.go:64] FLAG: --storage-driver-secure="false" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757995 2580 flags.go:64] FLAG: --storage-driver-table="stats" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.757998 2580 flags.go:64] FLAG: --storage-driver-user="root" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758001 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758004 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758007 2580 flags.go:64] FLAG: --system-cgroups="" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758010 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758015 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758017 2580 flags.go:64] FLAG: --tls-cert-file="" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758020 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758024 2580 flags.go:64] FLAG: --tls-min-version="" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758027 2580 flags.go:64] FLAG: --tls-private-key-file="" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758030 2580 flags.go:64] FLAG: --topology-manager-policy="none" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758033 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 17 15:55:41.764487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758036 2580 flags.go:64] FLAG: --topology-manager-scope="container" Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758039 2580 flags.go:64] FLAG: --v="2" Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758043 2580 flags.go:64] FLAG: --version="false" Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758047 2580 flags.go:64] FLAG: --vmodule="" Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758051 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.758054 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758173 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758177 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758179 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758182 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758185 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758187 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758190 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758193 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758196 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758199 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758202 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758205 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758207 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758210 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 15:55:41.765123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758213 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758215 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758218 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758221 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758224 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758228 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758231 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758233 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758236 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758242 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758245 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758248 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758250 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758253 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758255 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758258 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758261 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758264 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758268 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758271 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 15:55:41.765619 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758273 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758276 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758279 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758281 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758284 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758286 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758289 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758291 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758294 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758297 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758299 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758302 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758304 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758307 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758309 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758313 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758317 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758320 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758323 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758325 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 15:55:41.766101 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758328 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758332 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758334 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758337 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758340 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758343 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758345 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758348 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758351 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758354 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758358 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758361 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758363 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758366 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758368 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758371 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758373 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758376 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758379 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758382 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 15:55:41.766650 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758384 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758388 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758404 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758407 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758410 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758413 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758415 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758418 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758420 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758423 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758425 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758428 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758431 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758435 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758438 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758440 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758443 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758446 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758448 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 15:55:41.767217 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758451 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758454 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758456 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758461 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758463 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.758466 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.759166 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.766288 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.766303 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.766322 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.766334 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766388 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766408 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766412 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766415 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 15:55:41.767791 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766417 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766420 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766423 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766426 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766429 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766431 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766434 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766438 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766442 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766445 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766449 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766453 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766456 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766459 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766462 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766465 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766468 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766472 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766475 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 15:55:41.768160 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766477 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766480 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766482 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766486 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766490 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766493 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766503 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766507 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766510 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766513 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766515 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766518 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766521 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766524 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766527 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766530 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766532 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766535 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766537 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 15:55:41.768642 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766540 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766542 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766545 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766548 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766550 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766553 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766556 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766558 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766561 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766564 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766566 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766570 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766572 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766575 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766578 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766580 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766583 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766585 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766588 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 15:55:41.769103 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766591 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766595 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766597 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766600 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766602 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766605 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766608 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766610 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766613 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766615 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766618 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766620 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766623 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766625 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766627 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766630 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766632 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766635 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766637 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766640 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 15:55:41.769603 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766643 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766645 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766647 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766650 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766654 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766656 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766659 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766662 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766665 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766667 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766670 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766672 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766675 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766677 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766681 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766683 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766686 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 15:55:41.770123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766688 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.766693 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766798 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766802 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766804 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766807 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766809 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766812 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766814 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766817 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766820 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766822 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766825 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766827 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766830 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 17 15:55:41.770570 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766832 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766835 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766839 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766842 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766846 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766849 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766852 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766855 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766858 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766861 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766864 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766867 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766870 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766873 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766876 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766879 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766882 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766884 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766887 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 17 15:55:41.770939 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766889 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766892 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766894 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766897 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766899 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766902 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766904 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766907 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766910 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766912 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766915 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766917 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766920 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766922 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766925 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766927 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766930 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766933 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766937 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766954 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 17 15:55:41.771388 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766958 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766961 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766964 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766967 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766970 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766973 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766976 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766978 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766981 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766984 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766986 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766989 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766991 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766994 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.766997 2580 feature_gate.go:328] unrecognized feature gate: Example2 Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767000 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767002 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767005 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767008 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767010 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 17 15:55:41.771879 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767013 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767016 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767018 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767021 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767023 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767026 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767028 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767031 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767033 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767036 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767038 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767041 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767043 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767046 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767048 2580 feature_gate.go:328] unrecognized feature gate: Example Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767050 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767053 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767056 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767058 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767061 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 17 15:55:41.772354 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767064 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767066 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767069 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767072 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767074 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767076 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:41.767079 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.767083 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.767172 2580 server.go:962] "Client rotation is on, will bootstrap in background" Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.769371 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.770087 2580 server.go:1019] "Starting client certificate rotation" Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.770183 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.770219 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 17 15:55:41.772857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.772565 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 15:55:41.773587 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.773571 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 17 15:55:41.782476 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.782461 2580 log.go:25] "Validated CRI v1 runtime API" Sep 17 15:55:41.784770 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.784755 2580 log.go:25] "Validated CRI v1 image API" Sep 17 15:55:41.785619 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.785600 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 17 15:55:41.789821 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.789803 2580 fs.go:135] Filesystem UUIDs: map[0eca8467-1a37-4d1e-8b47-545376a8d4b4:/dev/nvme0n1p3 1f0df4ee-6d66-4593-84cb-3eeab287084e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Sep 17 15:55:41.789886 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.789819 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 17 15:55:41.795866 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.795694 2580 manager.go:216] Machine: {Timestamp:2026-09-17 15:55:41.794450897 +0000 UTC m=+0.302517940 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100778 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21aa912b99f2b89e8276866efec70a SystemUUID:ec21aa91-2b99-f2b8-9e82-76866efec70a BootID:e454a362-e710-4f56-86b8-f42340079b52 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f6:2e:80:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f6:2e:80:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:c4:74:b1:38:47 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 17 15:55:41.795866 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.795859 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 17 15:55:41.795991 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.795928 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 17 15:55:41.798793 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.798771 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 17 15:55:41.798946 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.798797 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-222.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 17 15:55:41.800143 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.800131 2580 topology_manager.go:138] "Creating topology manager with none policy" Sep 17 15:55:41.800208 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.800148 2580 container_manager_linux.go:309] "Creating device plugin manager" Sep 17 15:55:41.800208 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.800160 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 15:55:41.800807 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.800796 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 17 15:55:41.800887 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.800877 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 17 15:55:41.804774 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.804759 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 17 15:55:41.804898 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.804887 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 17 15:55:41.804954 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.804925 2580 kubelet.go:486] "Attempting to sync node with API server" Sep 17 15:55:41.804954 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.804939 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 17 15:55:41.805050 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.804959 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 17 15:55:41.805050 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.804974 2580 kubelet.go:398] "Adding apiserver pod source" Sep 17 15:55:41.805050 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.804985 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 17 15:55:41.805328 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.805314 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 15:55:41.805405 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.805338 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 17 15:55:41.805509 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.805488 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 15:55:41.807420 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.807390 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 17 15:55:41.810023 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.810009 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 17 15:55:41.810090 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.810039 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 17 15:55:41.811132 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811119 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 17 15:55:41.811132 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811135 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 17 15:55:41.811217 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811141 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 17 15:55:41.811217 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811146 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 17 15:55:41.811217 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811155 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 17 15:55:41.811217 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811160 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 17 15:55:41.811217 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811182 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 17 15:55:41.811217 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811188 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 17 15:55:41.811217 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811196 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 17 15:55:41.811217 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811202 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 17 15:55:41.811217 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811210 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 17 15:55:41.811217 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811215 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 17 15:55:41.811906 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811896 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 17 15:55:41.811906 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.811905 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 17 15:55:41.815493 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.815480 2580 server.go:1268] "Started kubelet" Sep 17 15:55:41.818860 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.818795 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 17 15:55:41.819341 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.819275 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 17 15:55:41.819438 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.819383 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 17 15:55:41.819345 ip-10-0-128-222 systemd[1]: Started Kubernetes Kubelet. Sep 17 15:55:41.821718 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.821702 2580 server.go:310] "Adding debug handlers to kubelet server" Sep 17 15:55:41.821904 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.821776 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 17 15:55:41.823171 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:41.823150 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 17 15:55:41.823267 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:41.823214 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-222.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 17 15:55:41.823362 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.823336 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-222.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 17 15:55:41.824002 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:41.823982 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 17 15:55:41.824153 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:41.823356 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-222.ec2.internal.18d626c7df7177d7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-222.ec2.internal,UID:ip-10-0-128-222.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-222.ec2.internal,},FirstTimestamp:2026-09-17 15:55:41.815453655 +0000 UTC m=+0.323520698,LastTimestamp:2026-09-17 15:55:41.815453655 +0000 UTC m=+0.323520698,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-222.ec2.internal,}" Sep 17 15:55:41.828629 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.828608 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 17 15:55:41.829205 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.829190 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 17 15:55:41.829311 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:41.829283 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-222.ec2.internal\" not found" Sep 17 15:55:41.829830 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.829813 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 17 15:55:41.829830 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.829823 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 17 15:55:41.829973 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.829838 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 17 15:55:41.829973 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.829901 2580 reconstruct.go:97] "Volume reconstruction finished" Sep 17 15:55:41.829973 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.829908 2580 reconciler.go:29] "Reconciler: start to sync state" Sep 17 15:55:41.829973 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.829909 2580 factory.go:55] Registering systemd factory Sep 17 15:55:41.829973 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.829947 2580 factory.go:223] Registration of the systemd container factory successfully Sep 17 15:55:41.830161 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.830130 2580 factory.go:153] Registering CRI-O factory Sep 17 15:55:41.830161 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.830144 2580 factory.go:223] Registration of the crio container factory successfully Sep 17 15:55:41.830253 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.830192 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 17 15:55:41.830253 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.830209 2580 factory.go:103] Registering Raw factory Sep 17 15:55:41.830253 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.830222 2580 manager.go:1196] Started watching for new ooms in manager Sep 17 15:55:41.830614 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.830601 2580 manager.go:349] Starting recovery of all containers Sep 17 15:55:41.832458 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:41.832420 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 17 15:55:41.832554 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:41.832490 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-222.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 17 15:55:41.840625 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.840608 2580 manager.go:354] Recovery completed Sep 17 15:55:41.844311 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.844297 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 15:55:41.846059 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.846037 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientMemory" Sep 17 15:55:41.846118 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.846074 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 15:55:41.846118 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.846089 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientPID" Sep 17 15:55:41.847338 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.847273 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 17 15:55:41.847338 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.847284 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 17 15:55:41.847338 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.847302 2580 state_mem.go:36] "Initialized new in-memory state store" Sep 17 15:55:41.847338 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.847305 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 15:55:41.848028 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.848012 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientMemory" Sep 17 15:55:41.848095 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.848041 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 15:55:41.848095 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.848053 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientPID" Sep 17 15:55:41.848641 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.848627 2580 policy_none.go:49] "None policy: Start" Sep 17 15:55:41.848641 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.848642 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 17 15:55:41.848721 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.848652 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 17 15:55:41.849612 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.849591 2580 policy_none.go:47] "Start" Sep 17 15:55:41.856464 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.856441 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zskx4" Sep 17 15:55:41.862028 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:41.861928 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-222.ec2.internal.18d626c7e14487ef default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-222.ec2.internal,UID:ip-10-0-128-222.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-222.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-222.ec2.internal,},FirstTimestamp:2026-09-17 15:55:41.846063087 +0000 UTC m=+0.354130132,LastTimestamp:2026-09-17 15:55:41.846063087 +0000 UTC m=+0.354130132,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-222.ec2.internal,}" Sep 17 15:55:41.865043 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.863734 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zskx4" Sep 17 15:55:41.876939 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.876925 2580 manager.go:337] "Starting Device Plugin manager" Sep 17 15:55:41.877027 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:41.876971 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 17 15:55:41.877027 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.876984 2580 server.go:87] "Starting device plugin registration server" Sep 17 15:55:41.877276 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.877262 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 17 15:55:41.877345 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.877276 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 17 15:55:41.877435 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.877411 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 17 15:55:41.877504 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.877492 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 17 15:55:41.877504 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.877503 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 17 15:55:41.877896 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:41.877874 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 17 15:55:41.877993 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:41.877921 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-222.ec2.internal\" not found" Sep 17 15:55:41.893421 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.893387 2580 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2612 into the system container \"/system.slice\": " Sep 17 15:55:41.978305 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.978285 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 15:55:41.979229 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.979213 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientMemory" Sep 17 15:55:41.979302 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.979243 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 15:55:41.979302 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.979256 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientPID" Sep 17 15:55:41.979302 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.979279 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-222.ec2.internal" Sep 17 15:55:41.987831 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:41.987817 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-222.ec2.internal" Sep 17 15:55:41.987873 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:41.987841 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-222.ec2.internal\": node \"ip-10-0-128-222.ec2.internal\" not found" Sep 17 15:55:42.014115 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:42.014094 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-222.ec2.internal\" not found" Sep 17 15:55:42.081040 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.081007 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 17 15:55:42.082348 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.082325 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 17 15:55:42.082467 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.082373 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 17 15:55:42.082467 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.082410 2580 kubelet.go:2448] "Starting kubelet main sync loop" Sep 17 15:55:42.082467 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:42.082456 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 17 15:55:42.085087 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.085070 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 15:55:42.115122 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:42.115102 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-222.ec2.internal\" not found" Sep 17 15:55:42.183490 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.183445 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-222.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal"] Sep 17 15:55:42.183569 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.183536 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 15:55:42.185048 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.185034 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientMemory" Sep 17 15:55:42.185126 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.185067 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 15:55:42.185126 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.185082 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientPID" Sep 17 15:55:42.186281 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.186266 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 15:55:42.186431 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.186416 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.186483 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.186444 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 15:55:42.187085 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.187063 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientMemory" Sep 17 15:55:42.187176 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.187107 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 15:55:42.187176 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.187122 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientPID" Sep 17 15:55:42.187176 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.187065 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientMemory" Sep 17 15:55:42.187176 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.187177 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 15:55:42.187353 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.187187 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientPID" Sep 17 15:55:42.188850 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.188833 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.188929 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.188856 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 17 15:55:42.189504 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.189489 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientMemory" Sep 17 15:55:42.189583 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.189513 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasNoDiskPressure" Sep 17 15:55:42.189583 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.189522 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeHasSufficientPID" Sep 17 15:55:42.214907 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:42.214891 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-222.ec2.internal\" not found" node="ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.215475 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:42.215459 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-222.ec2.internal\" not found" Sep 17 15:55:42.219035 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:42.219016 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-222.ec2.internal\" not found" node="ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.232251 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.232228 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02a1199f6936144f7915bb63aaf5c379-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal\" (UID: \"02a1199f6936144f7915bb63aaf5c379\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.232338 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.232251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/246f67986432aefd3f131bae29e7bad0-config\") pod \"kube-apiserver-proxy-ip-10-0-128-222.ec2.internal\" (UID: \"246f67986432aefd3f131bae29e7bad0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.232338 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.232267 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/02a1199f6936144f7915bb63aaf5c379-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal\" (UID: \"02a1199f6936144f7915bb63aaf5c379\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.315881 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:42.315834 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-222.ec2.internal\" not found" Sep 17 15:55:42.333188 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.333168 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/246f67986432aefd3f131bae29e7bad0-config\") pod \"kube-apiserver-proxy-ip-10-0-128-222.ec2.internal\" (UID: \"246f67986432aefd3f131bae29e7bad0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.333269 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.333197 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/02a1199f6936144f7915bb63aaf5c379-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal\" (UID: \"02a1199f6936144f7915bb63aaf5c379\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.333269 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.333215 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02a1199f6936144f7915bb63aaf5c379-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal\" (UID: \"02a1199f6936144f7915bb63aaf5c379\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.333371 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.333271 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02a1199f6936144f7915bb63aaf5c379-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal\" (UID: \"02a1199f6936144f7915bb63aaf5c379\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.333371 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.333290 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/02a1199f6936144f7915bb63aaf5c379-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal\" (UID: \"02a1199f6936144f7915bb63aaf5c379\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.333371 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.333270 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/246f67986432aefd3f131bae29e7bad0-config\") pod \"kube-apiserver-proxy-ip-10-0-128-222.ec2.internal\" (UID: \"246f67986432aefd3f131bae29e7bad0\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.416387 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:42.416341 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-222.ec2.internal\" not found" Sep 17 15:55:42.516935 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:42.516901 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-222.ec2.internal\" not found" Sep 17 15:55:42.520090 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.520074 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-222.ec2.internal" Sep 17 15:55:42.523429 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:42.523410 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" Sep 17 15:55:44.022145 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.022092 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-222.ec2.internal\" not found" Sep 17 15:55:44.079004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.078980 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 15:55:44.134424 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.134407 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-222.ec2.internal" Sep 17 15:55:44.150487 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.150465 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 17 15:55:44.152114 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.152102 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" Sep 17 15:55:44.174839 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.174823 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 17 15:55:44.175209 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.174941 2580 kubelet.go:3250] "Failed creating a mirror pod" err="Post \"https://a20f59d4ee05543f69000d2beb2a27aa-561b488962ba67b7.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/openshift-machine-config-operator/pods\": read tcp 10.0.128.222:60042->18.211.6.30:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" Sep 17 15:55:44.175209 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.174969 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 15:55:44.175209 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.174982 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 17 15:55:44.210110 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.210091 2580 apiserver.go:52] "Watching apiserver" Sep 17 15:55:44.233562 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.233544 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 17 15:55:44.234914 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.234897 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 17 15:55:44.236953 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.236933 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal","openshift-multus/network-metrics-daemon-8clfl","openshift-network-diagnostics/network-check-target-kkn7c","openshift-ovn-kubernetes/ovnkube-node-7zb8p","kube-system/global-pull-secret-syncer-8877v","kube-system/kube-apiserver-proxy-ip-10-0-128-222.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl","openshift-cluster-node-tuning-operator/tuned-89wdl","openshift-multus/multus-2slv4","openshift-multus/multus-additional-cni-plugins-sdkh6","openshift-network-operator/iptables-alerter-bvqtv","kube-system/konnectivity-agent-8sb8r","openshift-image-registry/node-ca-vlcft"] Sep 17 15:55:44.237855 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.237841 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 15:55:44.238264 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.238252 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.241096 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.241077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:44.241169 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.241146 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:55:44.242288 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.242275 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:44.242337 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.242321 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:55:44.243614 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.243602 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.244719 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.244702 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:44.244800 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.244763 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:55:44.245808 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.245793 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.246836 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.246821 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.247948 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.247935 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.249537 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.249522 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bvqtv" Sep 17 15:55:44.250815 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.250799 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 17 15:55:44.250881 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.250800 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 17 15:55:44.251015 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.251002 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 17 15:55:44.251050 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.251022 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 17 15:55:44.251105 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.251092 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 17 15:55:44.251145 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.251105 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 17 15:55:44.251369 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.251357 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8sb8r" Sep 17 15:55:44.252153 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.252140 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-k9w7s\"" Sep 17 15:55:44.252207 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.252143 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 17 15:55:44.252207 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.252168 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 17 15:55:44.252289 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.252155 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wfxkg\"" Sep 17 15:55:44.252665 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.252650 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 17 15:55:44.253204 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.253191 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vlcft" Sep 17 15:55:44.253382 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.253362 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-76pft\"" Sep 17 15:55:44.256393 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.256378 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 17 15:55:44.257820 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.257799 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 17 15:55:44.257820 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.257809 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lhnkc\"" Sep 17 15:55:44.257922 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.257800 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 17 15:55:44.257922 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.257806 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 17 15:55:44.257987 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.257797 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 17 15:55:44.258207 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.258191 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 17 15:55:44.258312 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.258298 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 17 15:55:44.258412 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.258398 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 17 15:55:44.258412 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.258397 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 17 15:55:44.259139 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.259125 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 17 15:55:44.259628 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.259606 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 17 15:55:44.260969 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.260948 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9m8hk\"" Sep 17 15:55:44.261052 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.260979 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 17 15:55:44.261052 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.260993 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 17 15:55:44.261052 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.261018 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 17 15:55:44.261052 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.261034 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r8m68\"" Sep 17 15:55:44.267805 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.260984 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 17 15:55:44.267805 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.260948 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fz5j4\"" Sep 17 15:55:44.267805 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.261173 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8f7rk\"" Sep 17 15:55:44.267805 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.261320 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 17 15:55:44.267805 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.262254 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 17 15:55:44.276036 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.275978 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-16 15:50:43 +0000 UTC" deadline="2028-05-12 00:19:20.874468146 +0000 UTC" Sep 17 15:55:44.276036 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.276014 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14456h23m36.598456912s" Sep 17 15:55:44.308098 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.307010 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2jvt6" Sep 17 15:55:44.321970 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.321953 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2jvt6" Sep 17 15:55:44.335866 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.335849 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 17 15:55:44.340942 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.340924 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-socket-dir\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.341034 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.340948 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbsnj\" (UniqueName: \"kubernetes.io/projected/edf5c407-8316-462e-b452-f867f1f5f301-kube-api-access-lbsnj\") pod \"node-ca-vlcft\" (UID: \"edf5c407-8316-462e-b452-f867f1f5f301\") " pod="openshift-image-registry/node-ca-vlcft" Sep 17 15:55:44.341034 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.340963 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-kubelet\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.341034 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.340978 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-systemd-units\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.341034 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.340998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-ovnkube-script-lib\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.341034 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341024 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9x99\" (UniqueName: \"kubernetes.io/projected/05ed94c8-e908-40bc-abf2-b770ebc1344a-kube-api-access-r9x99\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:44.341253 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341077 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-var-lib-kubelet\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.341253 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341148 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wj79\" (UniqueName: \"kubernetes.io/projected/4c35a0b6-2eb5-4459-80ef-012dda955f68-kube-api-access-8wj79\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.341253 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341183 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edf5c407-8316-462e-b452-f867f1f5f301-host\") pod \"node-ca-vlcft\" (UID: \"edf5c407-8316-462e-b452-f867f1f5f301\") " pod="openshift-image-registry/node-ca-vlcft" Sep 17 15:55:44.341253 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341206 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-run-systemd\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.341462 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341254 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-run-ovn\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.341462 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341303 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-run-k8s-cni-cncf-io\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.341462 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341330 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m74q4\" (UniqueName: \"kubernetes.io/projected/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-kube-api-access-m74q4\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.341462 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341354 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-multus-daemon-config\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.341462 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-host\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.341462 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-sys-fs\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.341462 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341424 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:44.341462 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-sys\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341469 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfjz5\" (UniqueName: \"kubernetes.io/projected/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-kube-api-access-sfjz5\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-tuned\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341517 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-var-lib-cni-bin\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341542 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/87ffca99-52b2-48df-bd31-35847f0524e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341565 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-etc-selinux\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341590 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb8ft\" (UniqueName: \"kubernetes.io/projected/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-kube-api-access-hb8ft\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341614 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-etc-openvswitch\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341636 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-cni-bin\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341659 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-ovn-node-metrics-cert\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341681 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-system-cni-dir\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341703 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-lib-modules\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341741 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-cni-binary-copy\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341771 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-run-netns\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-multus-conf-dir\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341817 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87ffca99-52b2-48df-bd31-35847f0524e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.341871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341853 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6039d184-66c2-4e53-b104-6fa1b4717dff-iptables-alerter-script\") pod \"iptables-alerter-bvqtv\" (UID: \"6039d184-66c2-4e53-b104-6fa1b4717dff\") " pod="openshift-network-operator/iptables-alerter-bvqtv" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341881 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-slash\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341903 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-run-ovn-kubernetes\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341926 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6lmz\" (UniqueName: \"kubernetes.io/projected/87ffca99-52b2-48df-bd31-35847f0524e6-kube-api-access-w6lmz\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341952 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2be6f78e-ee45-40c6-ba36-de1d778ff97f-konnectivity-ca\") pod \"konnectivity-agent-8sb8r\" (UID: \"2be6f78e-ee45-40c6-ba36-de1d778ff97f\") " pod="kube-system/konnectivity-agent-8sb8r" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.341973 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/609285b3-0b12-4a2c-a723-afb1861e75a4-dbus\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342006 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-kubernetes\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342031 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-sysctl-d\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342061 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-registration-dir\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/edf5c407-8316-462e-b452-f867f1f5f301-serviceca\") pod \"node-ca-vlcft\" (UID: \"edf5c407-8316-462e-b452-f867f1f5f301\") " pod="openshift-image-registry/node-ca-vlcft" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342106 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-node-log\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342130 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342152 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-device-dir\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342174 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-sysconfig\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342195 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c35a0b6-2eb5-4459-80ef-012dda955f68-tmp\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342235 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-cni-netd\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.342574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342259 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-ovnkube-config\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342292 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-etc-kubernetes\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342354 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-modprobe-d\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-sysctl-conf\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342407 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-log-socket\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342435 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87ffca99-52b2-48df-bd31-35847f0524e6-system-cni-dir\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342459 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x5v6\" (UniqueName: \"kubernetes.io/projected/6039d184-66c2-4e53-b104-6fa1b4717dff-kube-api-access-6x5v6\") pod \"iptables-alerter-bvqtv\" (UID: \"6039d184-66c2-4e53-b104-6fa1b4717dff\") " pod="openshift-network-operator/iptables-alerter-bvqtv" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342484 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdfhg\" (UniqueName: \"kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg\") pod \"network-check-target-kkn7c\" (UID: \"af1e77ed-0bbf-419b-8a67-029c77b67873\") " pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342522 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-systemd\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342545 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-var-lib-openvswitch\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342578 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-multus-socket-dir-parent\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342600 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-var-lib-cni-multus\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342623 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6039d184-66c2-4e53-b104-6fa1b4717dff-host-slash\") pod \"iptables-alerter-bvqtv\" (UID: \"6039d184-66c2-4e53-b104-6fa1b4717dff\") " pod="openshift-network-operator/iptables-alerter-bvqtv" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342646 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/609285b3-0b12-4a2c-a723-afb1861e75a4-kubelet-config\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342681 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.343147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342704 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-env-overrides\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.343631 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342736 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-var-lib-kubelet\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.343631 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342766 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87ffca99-52b2-48df-bd31-35847f0524e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.343631 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342808 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/87ffca99-52b2-48df-bd31-35847f0524e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.343631 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342824 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-run-openvswitch\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.343631 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342843 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-multus-cni-dir\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.343631 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342877 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87ffca99-52b2-48df-bd31-35847f0524e6-cnibin\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.343631 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342908 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-run\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.343631 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342924 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-run-netns\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.343631 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342944 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-cnibin\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.343631 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342958 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-hostroot\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.343631 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342973 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-run-multus-certs\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.343631 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.342993 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2be6f78e-ee45-40c6-ba36-de1d778ff97f-agent-certs\") pod \"konnectivity-agent-8sb8r\" (UID: \"2be6f78e-ee45-40c6-ba36-de1d778ff97f\") " pod="kube-system/konnectivity-agent-8sb8r" Sep 17 15:55:44.443489 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443467 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfjz5\" (UniqueName: \"kubernetes.io/projected/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-kube-api-access-sfjz5\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.443489 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443496 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-tuned\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.443645 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443514 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-var-lib-cni-bin\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.443645 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/87ffca99-52b2-48df-bd31-35847f0524e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.443645 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443551 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-etc-selinux\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.443645 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443573 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb8ft\" (UniqueName: \"kubernetes.io/projected/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-kube-api-access-hb8ft\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.443645 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-etc-openvswitch\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.443645 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443619 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-cni-bin\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.443645 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443615 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-var-lib-cni-bin\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.443645 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-ovn-node-metrics-cert\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443668 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-system-cni-dir\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443682 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-etc-openvswitch\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443690 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-lib-modules\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443715 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-cni-binary-copy\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-run-netns\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443778 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-multus-conf-dir\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87ffca99-52b2-48df-bd31-35847f0524e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6039d184-66c2-4e53-b104-6fa1b4717dff-iptables-alerter-script\") pod \"iptables-alerter-bvqtv\" (UID: \"6039d184-66c2-4e53-b104-6fa1b4717dff\") " pod="openshift-network-operator/iptables-alerter-bvqtv" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443840 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-slash\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:44.443846 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod246f67986432aefd3f131bae29e7bad0.slice/crio-6545b4adc6d224f11de6c68618d28cb3f2eb7ffa492bbb867c00be5b7797f111 WatchSource:0}: Error finding container 6545b4adc6d224f11de6c68618d28cb3f2eb7ffa492bbb867c00be5b7797f111: Status 404 returned error can't find the container with id 6545b4adc6d224f11de6c68618d28cb3f2eb7ffa492bbb867c00be5b7797f111 Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-run-ovn-kubernetes\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443885 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6lmz\" (UniqueName: \"kubernetes.io/projected/87ffca99-52b2-48df-bd31-35847f0524e6-kube-api-access-w6lmz\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2be6f78e-ee45-40c6-ba36-de1d778ff97f-konnectivity-ca\") pod \"konnectivity-agent-8sb8r\" (UID: \"2be6f78e-ee45-40c6-ba36-de1d778ff97f\") " pod="kube-system/konnectivity-agent-8sb8r" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443925 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/609285b3-0b12-4a2c-a723-afb1861e75a4-dbus\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443947 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-kubernetes\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443952 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-system-cni-dir\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.444019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443969 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-sysctl-d\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.443992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-registration-dir\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444000 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-cni-bin\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/edf5c407-8316-462e-b452-f867f1f5f301-serviceca\") pod \"node-ca-vlcft\" (UID: \"edf5c407-8316-462e-b452-f867f1f5f301\") " pod="openshift-image-registry/node-ca-vlcft" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444031 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-node-log\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444046 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-run-netns\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444050 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-device-dir\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444098 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444112 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-sysconfig\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444119 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-etc-selinux\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c35a0b6-2eb5-4459-80ef-012dda955f68-tmp\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444142 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-multus-conf-dir\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-cni-netd\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-ovnkube-config\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444184 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-device-dir\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-etc-kubernetes\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.444794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444410 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-registration-dir\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444321 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/609285b3-0b12-4a2c-a723-afb1861e75a4-dbus\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444328 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-sysconfig\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444352 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/87ffca99-52b2-48df-bd31-35847f0524e6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444368 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-cni-binary-copy\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444362 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444494 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-etc-kubernetes\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444504 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-lib-modules\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.444517 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444553 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-node-log\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444288 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-cni-netd\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.444576 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs podName:05ed94c8-e908-40bc-abf2-b770ebc1344a nodeName:}" failed. No retries permitted until 2026-09-17 15:55:44.944556626 +0000 UTC m=+2.047802010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs") pod "network-metrics-daemon-8clfl" (UID: "05ed94c8-e908-40bc-abf2-b770ebc1344a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444617 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-run-ovn-kubernetes\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444672 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-slash\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444764 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-sysctl-d\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444809 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-kubernetes\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444435 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-modprobe-d\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.445657 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-sysctl-conf\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-log-socket\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444890 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/edf5c407-8316-462e-b452-f867f1f5f301-serviceca\") pod \"node-ca-vlcft\" (UID: \"edf5c407-8316-462e-b452-f867f1f5f301\") " pod="openshift-image-registry/node-ca-vlcft" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444902 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87ffca99-52b2-48df-bd31-35847f0524e6-system-cni-dir\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x5v6\" (UniqueName: \"kubernetes.io/projected/6039d184-66c2-4e53-b104-6fa1b4717dff-kube-api-access-6x5v6\") pod \"iptables-alerter-bvqtv\" (UID: \"6039d184-66c2-4e53-b104-6fa1b4717dff\") " pod="openshift-network-operator/iptables-alerter-bvqtv" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444939 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-modprobe-d\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdfhg\" (UniqueName: \"kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg\") pod \"network-check-target-kkn7c\" (UID: \"af1e77ed-0bbf-419b-8a67-029c77b67873\") " pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444940 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-ovnkube-config\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-systemd\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445004 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87ffca99-52b2-48df-bd31-35847f0524e6-system-cni-dir\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445007 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-var-lib-openvswitch\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445035 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-sysctl-conf\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-multus-socket-dir-parent\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-var-lib-cni-multus\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445080 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-systemd\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445096 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-multus-socket-dir-parent\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6039d184-66c2-4e53-b104-6fa1b4717dff-host-slash\") pod \"iptables-alerter-bvqtv\" (UID: \"6039d184-66c2-4e53-b104-6fa1b4717dff\") " pod="openshift-network-operator/iptables-alerter-bvqtv" Sep 17 15:55:44.446556 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445043 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-var-lib-openvswitch\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.444946 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-log-socket\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445147 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-var-lib-cni-multus\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/609285b3-0b12-4a2c-a723-afb1861e75a4-kubelet-config\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445188 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6039d184-66c2-4e53-b104-6fa1b4717dff-host-slash\") pod \"iptables-alerter-bvqtv\" (UID: \"6039d184-66c2-4e53-b104-6fa1b4717dff\") " pod="openshift-network-operator/iptables-alerter-bvqtv" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-env-overrides\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445239 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-var-lib-kubelet\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445250 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/609285b3-0b12-4a2c-a723-afb1861e75a4-kubelet-config\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445260 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87ffca99-52b2-48df-bd31-35847f0524e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445272 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445288 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/87ffca99-52b2-48df-bd31-35847f0524e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445309 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-run-openvswitch\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445329 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-multus-cni-dir\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445331 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2be6f78e-ee45-40c6-ba36-de1d778ff97f-konnectivity-ca\") pod \"konnectivity-agent-8sb8r\" (UID: \"2be6f78e-ee45-40c6-ba36-de1d778ff97f\") " pod="kube-system/konnectivity-agent-8sb8r" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445350 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87ffca99-52b2-48df-bd31-35847f0524e6-cnibin\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-run\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.447375 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445413 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-var-lib-kubelet\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-run-netns\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445438 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-cnibin\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445448 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-run-netns\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445458 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-hostroot\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445478 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-run-multus-certs\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445496 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87ffca99-52b2-48df-bd31-35847f0524e6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445533 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-run-openvswitch\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445586 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-multus-cni-dir\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445629 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-cnibin\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445645 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87ffca99-52b2-48df-bd31-35847f0524e6-cni-binary-copy\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445659 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-env-overrides\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445667 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-hostroot\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445499 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2be6f78e-ee45-40c6-ba36-de1d778ff97f-agent-certs\") pod \"konnectivity-agent-8sb8r\" (UID: \"2be6f78e-ee45-40c6-ba36-de1d778ff97f\") " pod="kube-system/konnectivity-agent-8sb8r" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-socket-dir\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445709 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-run-multus-certs\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445717 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-run\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445723 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbsnj\" (UniqueName: \"kubernetes.io/projected/edf5c407-8316-462e-b452-f867f1f5f301-kube-api-access-lbsnj\") pod \"node-ca-vlcft\" (UID: \"edf5c407-8316-462e-b452-f867f1f5f301\") " pod="openshift-image-registry/node-ca-vlcft" Sep 17 15:55:44.448268 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445748 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-kubelet\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445768 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-systemd-units\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-ovnkube-script-lib\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445806 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-socket-dir\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445809 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9x99\" (UniqueName: \"kubernetes.io/projected/05ed94c8-e908-40bc-abf2-b770ebc1344a-kube-api-access-r9x99\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445850 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-var-lib-kubelet\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445880 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/87ffca99-52b2-48df-bd31-35847f0524e6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445885 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wj79\" (UniqueName: \"kubernetes.io/projected/4c35a0b6-2eb5-4459-80ef-012dda955f68-kube-api-access-8wj79\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445929 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-var-lib-kubelet\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edf5c407-8316-462e-b452-f867f1f5f301-host\") pod \"node-ca-vlcft\" (UID: \"edf5c407-8316-462e-b452-f867f1f5f301\") " pod="openshift-image-registry/node-ca-vlcft" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445959 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-run-systemd\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445965 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/edf5c407-8316-462e-b452-f867f1f5f301-host\") pod \"node-ca-vlcft\" (UID: \"edf5c407-8316-462e-b452-f867f1f5f301\") " pod="openshift-image-registry/node-ca-vlcft" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-run-ovn\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446039 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-run-k8s-cni-cncf-io\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m74q4\" (UniqueName: \"kubernetes.io/projected/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-kube-api-access-m74q4\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.445388 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87ffca99-52b2-48df-bd31-35847f0524e6-cnibin\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-multus-daemon-config\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.448986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446232 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6039d184-66c2-4e53-b104-6fa1b4717dff-iptables-alerter-script\") pod \"iptables-alerter-bvqtv\" (UID: \"6039d184-66c2-4e53-b104-6fa1b4717dff\") " pod="openshift-network-operator/iptables-alerter-bvqtv" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446226 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-host-kubelet\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446253 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-host\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-sys-fs\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446321 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-sys\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446401 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-run-systemd\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446432 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-sys\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446445 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-run-ovn\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446485 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c35a0b6-2eb5-4459-80ef-012dda955f68-host\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446494 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-host-run-k8s-cni-cncf-io\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446536 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-sys-fs\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446559 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-systemd-units\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.446654 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.446721 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret podName:609285b3-0b12-4a2c-a723-afb1861e75a4 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:44.946704903 +0000 UTC m=+2.049950283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret") pod "global-pull-secret-syncer-8877v" (UID: "609285b3-0b12-4a2c-a723-afb1861e75a4") : object "kube-system"/"original-pull-secret" not registered Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.446831 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-ovnkube-script-lib\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.447062 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-multus-daemon-config\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.448484 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4c35a0b6-2eb5-4459-80ef-012dda955f68-etc-tuned\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.449486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.448560 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2be6f78e-ee45-40c6-ba36-de1d778ff97f-agent-certs\") pod \"konnectivity-agent-8sb8r\" (UID: \"2be6f78e-ee45-40c6-ba36-de1d778ff97f\") " pod="kube-system/konnectivity-agent-8sb8r" Sep 17 15:55:44.450161 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.448570 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-ovn-node-metrics-cert\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.450161 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.448599 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c35a0b6-2eb5-4459-80ef-012dda955f68-tmp\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.452493 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.452477 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 15:55:44.458987 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.458970 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfjz5\" (UniqueName: \"kubernetes.io/projected/e7d7c2f1-e491-428d-98e5-573bb6cc97d0-kube-api-access-sfjz5\") pod \"ovnkube-node-7zb8p\" (UID: \"e7d7c2f1-e491-428d-98e5-573bb6cc97d0\") " pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.459060 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.459048 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb8ft\" (UniqueName: \"kubernetes.io/projected/25c06c3b-5780-45c9-8a82-c4cfd9d6a57d-kube-api-access-hb8ft\") pod \"aws-ebs-csi-driver-node-8v7dl\" (UID: \"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.460894 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.460871 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6lmz\" (UniqueName: \"kubernetes.io/projected/87ffca99-52b2-48df-bd31-35847f0524e6-kube-api-access-w6lmz\") pod \"multus-additional-cni-plugins-sdkh6\" (UID: \"87ffca99-52b2-48df-bd31-35847f0524e6\") " pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.461637 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.461624 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 15:55:44.461685 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.461641 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 15:55:44.461685 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.461653 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vdfhg for pod openshift-network-diagnostics/network-check-target-kkn7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:55:44.461753 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.461694 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg podName:af1e77ed-0bbf-419b-8a67-029c77b67873 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:44.961684609 +0000 UTC m=+2.064929987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vdfhg" (UniqueName: "kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg") pod "network-check-target-kkn7c" (UID: "af1e77ed-0bbf-419b-8a67-029c77b67873") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:55:44.463799 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.463781 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x5v6\" (UniqueName: \"kubernetes.io/projected/6039d184-66c2-4e53-b104-6fa1b4717dff-kube-api-access-6x5v6\") pod \"iptables-alerter-bvqtv\" (UID: \"6039d184-66c2-4e53-b104-6fa1b4717dff\") " pod="openshift-network-operator/iptables-alerter-bvqtv" Sep 17 15:55:44.465907 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.465888 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m74q4\" (UniqueName: \"kubernetes.io/projected/cb4be840-5ccd-4ea0-900c-ee3ad4111e15-kube-api-access-m74q4\") pod \"multus-2slv4\" (UID: \"cb4be840-5ccd-4ea0-900c-ee3ad4111e15\") " pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.470314 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.470298 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wj79\" (UniqueName: \"kubernetes.io/projected/4c35a0b6-2eb5-4459-80ef-012dda955f68-kube-api-access-8wj79\") pod \"tuned-89wdl\" (UID: \"4c35a0b6-2eb5-4459-80ef-012dda955f68\") " pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.470504 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.470486 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbsnj\" (UniqueName: \"kubernetes.io/projected/edf5c407-8316-462e-b452-f867f1f5f301-kube-api-access-lbsnj\") pod \"node-ca-vlcft\" (UID: \"edf5c407-8316-462e-b452-f867f1f5f301\") " pod="openshift-image-registry/node-ca-vlcft" Sep 17 15:55:44.471714 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.471696 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9x99\" (UniqueName: \"kubernetes.io/projected/05ed94c8-e908-40bc-abf2-b770ebc1344a-kube-api-access-r9x99\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:44.480021 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.480006 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 15:55:44.485123 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:44.485104 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a1199f6936144f7915bb63aaf5c379.slice/crio-841048ef0976b2f74601a2c5f553612471261feed58874ec5a7def412d1dacf7 WatchSource:0}: Error finding container 841048ef0976b2f74601a2c5f553612471261feed58874ec5a7def412d1dacf7: Status 404 returned error can't find the container with id 841048ef0976b2f74601a2c5f553612471261feed58874ec5a7def412d1dacf7 Sep 17 15:55:44.493167 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.493130 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" event={"ID":"02a1199f6936144f7915bb63aaf5c379","Type":"ContainerStarted","Data":"841048ef0976b2f74601a2c5f553612471261feed58874ec5a7def412d1dacf7"} Sep 17 15:55:44.494197 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.494174 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-222.ec2.internal" event={"ID":"246f67986432aefd3f131bae29e7bad0","Type":"ContainerStarted","Data":"6545b4adc6d224f11de6c68618d28cb3f2eb7ffa492bbb867c00be5b7797f111"} Sep 17 15:55:44.569667 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.569624 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" Sep 17 15:55:44.576826 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:44.576807 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25c06c3b_5780_45c9_8a82_c4cfd9d6a57d.slice/crio-a48bcf559e348bffcc23b493adbded4376ecbd71cf369644cdeeaec9312294c3 WatchSource:0}: Error finding container a48bcf559e348bffcc23b493adbded4376ecbd71cf369644cdeeaec9312294c3: Status 404 returned error can't find the container with id a48bcf559e348bffcc23b493adbded4376ecbd71cf369644cdeeaec9312294c3 Sep 17 15:55:44.580898 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.580882 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:55:44.587395 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:44.587378 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7d7c2f1_e491_428d_98e5_573bb6cc97d0.slice/crio-2b702de734893b2d56e19185b5081503738212ab5dfe36fde7acf552a0e0b8a1 WatchSource:0}: Error finding container 2b702de734893b2d56e19185b5081503738212ab5dfe36fde7acf552a0e0b8a1: Status 404 returned error can't find the container with id 2b702de734893b2d56e19185b5081503738212ab5dfe36fde7acf552a0e0b8a1 Sep 17 15:55:44.600077 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.600060 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-89wdl" Sep 17 15:55:44.606600 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:44.606581 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c35a0b6_2eb5_4459_80ef_012dda955f68.slice/crio-606617455ebb298316a4b048980f1cc0ef1f5fe3cedef215b5d24c00695c1b98 WatchSource:0}: Error finding container 606617455ebb298316a4b048980f1cc0ef1f5fe3cedef215b5d24c00695c1b98: Status 404 returned error can't find the container with id 606617455ebb298316a4b048980f1cc0ef1f5fe3cedef215b5d24c00695c1b98 Sep 17 15:55:44.612385 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.612368 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2slv4" Sep 17 15:55:44.618421 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:44.618405 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb4be840_5ccd_4ea0_900c_ee3ad4111e15.slice/crio-bfc0e099ae5a38223fd2934ca66151b9f198220449279737a48650eee3489a0b WatchSource:0}: Error finding container bfc0e099ae5a38223fd2934ca66151b9f198220449279737a48650eee3489a0b: Status 404 returned error can't find the container with id bfc0e099ae5a38223fd2934ca66151b9f198220449279737a48650eee3489a0b Sep 17 15:55:44.627019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.627004 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-sdkh6" Sep 17 15:55:44.633098 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:44.633076 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87ffca99_52b2_48df_bd31_35847f0524e6.slice/crio-6f1b24fcbc448d8ffe2d61e0dc70eca5254c2ca707896ca2e9756edc669df7bb WatchSource:0}: Error finding container 6f1b24fcbc448d8ffe2d61e0dc70eca5254c2ca707896ca2e9756edc669df7bb: Status 404 returned error can't find the container with id 6f1b24fcbc448d8ffe2d61e0dc70eca5254c2ca707896ca2e9756edc669df7bb Sep 17 15:55:44.646056 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.646036 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bvqtv" Sep 17 15:55:44.652589 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:44.652571 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6039d184_66c2_4e53_b104_6fa1b4717dff.slice/crio-8bebb7375a44e2ece145c6ed5ca2b28405f15126444ca93e6121ad8ff10dd76d WatchSource:0}: Error finding container 8bebb7375a44e2ece145c6ed5ca2b28405f15126444ca93e6121ad8ff10dd76d: Status 404 returned error can't find the container with id 8bebb7375a44e2ece145c6ed5ca2b28405f15126444ca93e6121ad8ff10dd76d Sep 17 15:55:44.661373 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.661356 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8sb8r" Sep 17 15:55:44.666045 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.666025 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vlcft" Sep 17 15:55:44.667892 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:44.667867 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be6f78e_ee45_40c6_ba36_de1d778ff97f.slice/crio-93cd99a84e518d6c773397390bf0962c6e56d2519246d3ebcfbb35dc83ea29c7 WatchSource:0}: Error finding container 93cd99a84e518d6c773397390bf0962c6e56d2519246d3ebcfbb35dc83ea29c7: Status 404 returned error can't find the container with id 93cd99a84e518d6c773397390bf0962c6e56d2519246d3ebcfbb35dc83ea29c7 Sep 17 15:55:44.672674 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:55:44.672654 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedf5c407_8316_462e_b452_f867f1f5f301.slice/crio-c893f92d9b6984edcc6b82bca509342b7ebcbd03f9ea04b6c36ee225b0edd38b WatchSource:0}: Error finding container c893f92d9b6984edcc6b82bca509342b7ebcbd03f9ea04b6c36ee225b0edd38b: Status 404 returned error can't find the container with id c893f92d9b6984edcc6b82bca509342b7ebcbd03f9ea04b6c36ee225b0edd38b Sep 17 15:55:44.949578 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.949499 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:44.949578 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:44.949563 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:44.949773 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.949673 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:55:44.949773 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.949734 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs podName:05ed94c8-e908-40bc-abf2-b770ebc1344a nodeName:}" failed. No retries permitted until 2026-09-17 15:55:45.949716797 +0000 UTC m=+3.052962172 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs") pod "network-metrics-daemon-8clfl" (UID: "05ed94c8-e908-40bc-abf2-b770ebc1344a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:55:44.949773 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.949673 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 15:55:44.949927 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:44.949808 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret podName:609285b3-0b12-4a2c-a723-afb1861e75a4 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:45.949793272 +0000 UTC m=+3.053038646 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret") pod "global-pull-secret-syncer-8877v" (UID: "609285b3-0b12-4a2c-a723-afb1861e75a4") : object "kube-system"/"original-pull-secret" not registered Sep 17 15:55:45.050285 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.050190 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdfhg\" (UniqueName: \"kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg\") pod \"network-check-target-kkn7c\" (UID: \"af1e77ed-0bbf-419b-8a67-029c77b67873\") " pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:45.050565 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:45.050374 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 15:55:45.050565 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:45.050393 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 15:55:45.050565 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:45.050404 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vdfhg for pod openshift-network-diagnostics/network-check-target-kkn7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:55:45.050565 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:45.050461 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg podName:af1e77ed-0bbf-419b-8a67-029c77b67873 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:46.050448032 +0000 UTC m=+3.153693406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdfhg" (UniqueName: "kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg") pod "network-check-target-kkn7c" (UID: "af1e77ed-0bbf-419b-8a67-029c77b67873") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:55:45.324178 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.324049 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 15:50:44 +0000 UTC" deadline="2028-05-31 20:06:23.541795134 +0000 UTC" Sep 17 15:55:45.324178 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.324081 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14932h10m38.217718676s" Sep 17 15:55:45.463933 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.463668 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 15:55:45.491173 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.490749 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:45.491173 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:45.490865 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:55:45.514445 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.514373 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" event={"ID":"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d","Type":"ContainerStarted","Data":"a48bcf559e348bffcc23b493adbded4376ecbd71cf369644cdeeaec9312294c3"} Sep 17 15:55:45.529595 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.528951 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8sb8r" event={"ID":"2be6f78e-ee45-40c6-ba36-de1d778ff97f","Type":"ContainerStarted","Data":"93cd99a84e518d6c773397390bf0962c6e56d2519246d3ebcfbb35dc83ea29c7"} Sep 17 15:55:45.541786 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.541699 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vlcft" event={"ID":"edf5c407-8316-462e-b452-f867f1f5f301","Type":"ContainerStarted","Data":"c893f92d9b6984edcc6b82bca509342b7ebcbd03f9ea04b6c36ee225b0edd38b"} Sep 17 15:55:45.544615 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.544540 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bvqtv" event={"ID":"6039d184-66c2-4e53-b104-6fa1b4717dff","Type":"ContainerStarted","Data":"8bebb7375a44e2ece145c6ed5ca2b28405f15126444ca93e6121ad8ff10dd76d"} Sep 17 15:55:45.558278 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.558252 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdkh6" event={"ID":"87ffca99-52b2-48df-bd31-35847f0524e6","Type":"ContainerStarted","Data":"6f1b24fcbc448d8ffe2d61e0dc70eca5254c2ca707896ca2e9756edc669df7bb"} Sep 17 15:55:45.564823 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.564799 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2slv4" event={"ID":"cb4be840-5ccd-4ea0-900c-ee3ad4111e15","Type":"ContainerStarted","Data":"bfc0e099ae5a38223fd2934ca66151b9f198220449279737a48650eee3489a0b"} Sep 17 15:55:45.575451 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.575350 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-89wdl" event={"ID":"4c35a0b6-2eb5-4459-80ef-012dda955f68","Type":"ContainerStarted","Data":"606617455ebb298316a4b048980f1cc0ef1f5fe3cedef215b5d24c00695c1b98"} Sep 17 15:55:45.577121 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.577034 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" event={"ID":"e7d7c2f1-e491-428d-98e5-573bb6cc97d0","Type":"ContainerStarted","Data":"2b702de734893b2d56e19185b5081503738212ab5dfe36fde7acf552a0e0b8a1"} Sep 17 15:55:45.960574 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.960474 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:45.960730 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:45.960575 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:45.960730 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:45.960707 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 15:55:45.960843 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:45.960772 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret podName:609285b3-0b12-4a2c-a723-afb1861e75a4 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:47.960754217 +0000 UTC m=+5.063999602 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret") pod "global-pull-secret-syncer-8877v" (UID: "609285b3-0b12-4a2c-a723-afb1861e75a4") : object "kube-system"/"original-pull-secret" not registered Sep 17 15:55:45.961151 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:45.961128 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:55:45.961282 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:45.961184 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs podName:05ed94c8-e908-40bc-abf2-b770ebc1344a nodeName:}" failed. No retries permitted until 2026-09-17 15:55:47.961169042 +0000 UTC m=+5.064414408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs") pod "network-metrics-daemon-8clfl" (UID: "05ed94c8-e908-40bc-abf2-b770ebc1344a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:55:46.062252 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:46.061080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdfhg\" (UniqueName: \"kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg\") pod \"network-check-target-kkn7c\" (UID: \"af1e77ed-0bbf-419b-8a67-029c77b67873\") " pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:46.062252 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:46.061284 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 15:55:46.062252 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:46.061307 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 15:55:46.062252 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:46.061319 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vdfhg for pod openshift-network-diagnostics/network-check-target-kkn7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:55:46.062252 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:46.061376 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg podName:af1e77ed-0bbf-419b-8a67-029c77b67873 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:48.061358872 +0000 UTC m=+5.164604245 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdfhg" (UniqueName: "kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg") pod "network-check-target-kkn7c" (UID: "af1e77ed-0bbf-419b-8a67-029c77b67873") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:55:46.324293 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:46.324199 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-16 15:50:44 +0000 UTC" deadline="2028-06-27 10:28:05.500929354 +0000 UTC" Sep 17 15:55:46.324293 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:46.324246 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15570h32m19.17668745s" Sep 17 15:55:46.488149 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:46.488117 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:46.489028 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:46.489003 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:46.489179 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:46.489157 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:55:46.489354 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:46.489326 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:55:46.593107 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:46.592848 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 17 15:55:47.488365 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:47.488339 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:47.488886 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:47.488451 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:55:47.979721 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:47.979635 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:47.979721 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:47.979711 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:47.979965 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:47.979842 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:55:47.979965 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:47.979904 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs podName:05ed94c8-e908-40bc-abf2-b770ebc1344a nodeName:}" failed. No retries permitted until 2026-09-17 15:55:51.97988567 +0000 UTC m=+9.083131044 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs") pod "network-metrics-daemon-8clfl" (UID: "05ed94c8-e908-40bc-abf2-b770ebc1344a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:55:47.980374 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:47.980286 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 15:55:47.980374 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:47.980337 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret podName:609285b3-0b12-4a2c-a723-afb1861e75a4 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:51.980323927 +0000 UTC m=+9.083569290 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret") pod "global-pull-secret-syncer-8877v" (UID: "609285b3-0b12-4a2c-a723-afb1861e75a4") : object "kube-system"/"original-pull-secret" not registered Sep 17 15:55:48.081187 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:48.080565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdfhg\" (UniqueName: \"kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg\") pod \"network-check-target-kkn7c\" (UID: \"af1e77ed-0bbf-419b-8a67-029c77b67873\") " pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:48.081187 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:48.080771 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 15:55:48.081187 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:48.080794 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 15:55:48.081187 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:48.080807 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vdfhg for pod openshift-network-diagnostics/network-check-target-kkn7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:55:48.081187 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:48.080865 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg podName:af1e77ed-0bbf-419b-8a67-029c77b67873 nodeName:}" failed. No retries permitted until 2026-09-17 15:55:52.080848114 +0000 UTC m=+9.184093480 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdfhg" (UniqueName: "kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg") pod "network-check-target-kkn7c" (UID: "af1e77ed-0bbf-419b-8a67-029c77b67873") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:55:48.487795 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:48.487762 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:48.487969 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:48.487895 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:55:48.488318 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:48.488299 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:48.488433 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:48.488411 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:55:49.488651 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:49.488615 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:49.489086 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:49.488737 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:55:50.487971 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:50.487751 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:50.487971 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:50.487799 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:50.487971 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:50.487890 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:55:50.488287 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:50.487985 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:55:51.488617 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:51.488588 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:51.489187 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:51.488719 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:55:52.014173 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:52.014131 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:52.014364 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:52.014230 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:52.014364 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:52.014352 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:55:52.014482 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:52.014424 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs podName:05ed94c8-e908-40bc-abf2-b770ebc1344a nodeName:}" failed. No retries permitted until 2026-09-17 15:56:00.014404188 +0000 UTC m=+17.117649563 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs") pod "network-metrics-daemon-8clfl" (UID: "05ed94c8-e908-40bc-abf2-b770ebc1344a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:55:52.014482 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:52.014352 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 15:55:52.014591 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:52.014499 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret podName:609285b3-0b12-4a2c-a723-afb1861e75a4 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:00.014482711 +0000 UTC m=+17.117728072 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret") pod "global-pull-secret-syncer-8877v" (UID: "609285b3-0b12-4a2c-a723-afb1861e75a4") : object "kube-system"/"original-pull-secret" not registered Sep 17 15:55:52.115005 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:52.114953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdfhg\" (UniqueName: \"kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg\") pod \"network-check-target-kkn7c\" (UID: \"af1e77ed-0bbf-419b-8a67-029c77b67873\") " pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:52.115170 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:52.115041 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 15:55:52.115170 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:52.115062 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 15:55:52.115170 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:52.115075 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vdfhg for pod openshift-network-diagnostics/network-check-target-kkn7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:55:52.115170 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:52.115130 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg podName:af1e77ed-0bbf-419b-8a67-029c77b67873 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:00.115111991 +0000 UTC m=+17.218357367 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdfhg" (UniqueName: "kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg") pod "network-check-target-kkn7c" (UID: "af1e77ed-0bbf-419b-8a67-029c77b67873") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:55:52.488040 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:52.487549 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:52.488040 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:52.487803 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:55:52.488346 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:52.488321 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:52.488500 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:52.488479 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:55:53.488794 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:53.488754 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:53.489415 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:53.488864 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:55:54.487770 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:54.487735 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:54.487940 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:54.487743 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:54.487940 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:54.487850 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:55:54.488066 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:54.487937 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:55:55.488205 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:55.488179 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:55.488540 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:55.488319 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:55:56.487608 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:56.487570 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:56.487781 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:56.487616 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:56.487781 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:56.487731 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:55:56.487904 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:56.487853 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:55:57.487928 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:57.487895 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:57.488396 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:57.488022 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:55:58.487754 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:58.487717 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:55:58.487938 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:58.487848 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:55:58.487938 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:58.487911 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:55:58.488377 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:58.488024 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:55:59.487942 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:55:59.487906 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:55:59.488374 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:55:59.488037 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:00.068258 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:00.068224 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:00.068485 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:00.068289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:00.068485 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:00.068378 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 15:56:00.068485 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:00.068406 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:56:00.068485 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:00.068451 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret podName:609285b3-0b12-4a2c-a723-afb1861e75a4 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:16.068433146 +0000 UTC m=+33.171678523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret") pod "global-pull-secret-syncer-8877v" (UID: "609285b3-0b12-4a2c-a723-afb1861e75a4") : object "kube-system"/"original-pull-secret" not registered Sep 17 15:56:00.068485 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:00.068473 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs podName:05ed94c8-e908-40bc-abf2-b770ebc1344a nodeName:}" failed. No retries permitted until 2026-09-17 15:56:16.06846195 +0000 UTC m=+33.171707315 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs") pod "network-metrics-daemon-8clfl" (UID: "05ed94c8-e908-40bc-abf2-b770ebc1344a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:56:00.168872 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:00.168837 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdfhg\" (UniqueName: \"kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg\") pod \"network-check-target-kkn7c\" (UID: \"af1e77ed-0bbf-419b-8a67-029c77b67873\") " pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:00.169035 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:00.168982 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 15:56:00.169035 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:00.169003 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 15:56:00.169035 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:00.169012 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vdfhg for pod openshift-network-diagnostics/network-check-target-kkn7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:56:00.169155 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:00.169064 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg podName:af1e77ed-0bbf-419b-8a67-029c77b67873 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:16.16905099 +0000 UTC m=+33.272296364 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdfhg" (UniqueName: "kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg") pod "network-check-target-kkn7c" (UID: "af1e77ed-0bbf-419b-8a67-029c77b67873") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:56:00.488031 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:00.487938 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:00.488463 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:00.487938 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:00.488463 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:00.488089 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:00.488463 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:00.488156 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:01.488434 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:01.488400 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:01.489001 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:01.488518 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:02.488365 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:02.488329 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:02.488534 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:02.488338 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:02.488534 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:02.488473 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:02.488920 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:02.488528 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:03.489201 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:03.488832 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:03.489201 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:03.489116 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:03.618108 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:03.618072 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-89wdl" event={"ID":"4c35a0b6-2eb5-4459-80ef-012dda955f68","Type":"ContainerStarted","Data":"645099ae5a00e1ccdb67b667b19fd139e89bd51543529e73412e5af982ab09b7"} Sep 17 15:56:03.629998 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:03.629973 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 15:56:03.630519 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:03.630432 2580 generic.go:356] "Generic (PLEG): container finished" podID="e7d7c2f1-e491-428d-98e5-573bb6cc97d0" containerID="60b25efab516490040ea3c3397d4c702b234c132a49b0834fa2c8ec6f4b9db82" exitCode=1 Sep 17 15:56:03.630606 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:03.630520 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" event={"ID":"e7d7c2f1-e491-428d-98e5-573bb6cc97d0","Type":"ContainerStarted","Data":"2888e0dc9b74609bd963d08ef3d4aa913a0f56748b64df32c739b001e16fdaba"} Sep 17 15:56:03.630723 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:03.630629 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" event={"ID":"e7d7c2f1-e491-428d-98e5-573bb6cc97d0","Type":"ContainerDied","Data":"60b25efab516490040ea3c3397d4c702b234c132a49b0834fa2c8ec6f4b9db82"} Sep 17 15:56:03.630723 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:03.630686 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" event={"ID":"e7d7c2f1-e491-428d-98e5-573bb6cc97d0","Type":"ContainerStarted","Data":"19da3cbee0b6a95082418037c594c215703c40e159e5342a1117d50be96e3ea0"} Sep 17 15:56:03.632990 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:03.632969 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-222.ec2.internal" event={"ID":"246f67986432aefd3f131bae29e7bad0","Type":"ContainerStarted","Data":"d06d35344173ad053804825624441f6bae109a01cf712f06575c5c450a5bd779"} Sep 17 15:56:03.639171 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:03.639130 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-89wdl" podStartSLOduration=2.196716541 podStartE2EDuration="20.63911684s" totalImagesPullingTime="18.442400299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:03.639061878 +0000 UTC m=+20.742307260" watchObservedRunningTime="2026-09-17 15:56:03.63911684 +0000 UTC m=+20.742362221" Sep 17 15:56:04.487612 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.487435 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:04.487817 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.487436 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:04.487817 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:04.487716 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:04.487908 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:04.487827 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:04.637761 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.637728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2slv4" event={"ID":"cb4be840-5ccd-4ea0-900c-ee3ad4111e15","Type":"ContainerStarted","Data":"9bda7f1dbcbd68cf39500c40a3955ae9bfd760cde40f603502b1d79deea19684"} Sep 17 15:56:04.640898 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.640877 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 15:56:04.641329 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.641307 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" event={"ID":"e7d7c2f1-e491-428d-98e5-573bb6cc97d0","Type":"ContainerStarted","Data":"3eceb352cb393338c3ab3329ff6005f63c90105dfd9b76f3317a1dd0a070a67f"} Sep 17 15:56:04.641441 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.641387 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" event={"ID":"e7d7c2f1-e491-428d-98e5-573bb6cc97d0","Type":"ContainerStarted","Data":"ed2a47761f78cd5f01b9e7cf92c58ccbeee0777b05764d5284d3672d10590229"} Sep 17 15:56:04.641496 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.641443 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" event={"ID":"e7d7c2f1-e491-428d-98e5-573bb6cc97d0","Type":"ContainerStarted","Data":"8a28a36477a45b947e32aed7d2d343131806d6188a49cdfea98c3455a34f831c"} Sep 17 15:56:04.643073 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.643037 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" event={"ID":"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d","Type":"ContainerStarted","Data":"5de20ba22c9d449488852b968a98591953d13503fd80a594627816634885555e"} Sep 17 15:56:04.644598 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.644577 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8sb8r" event={"ID":"2be6f78e-ee45-40c6-ba36-de1d778ff97f","Type":"ContainerStarted","Data":"1a7dc6719c5e25aa4c14e1817e44804b82fa04a9c66a5f7344f761c70e4bfe8c"} Sep 17 15:56:04.646291 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.646267 2580 generic.go:356] "Generic (PLEG): container finished" podID="02a1199f6936144f7915bb63aaf5c379" containerID="8ca74c77c401b080f59d3ba3bc29b297bd6716353770b11a1917ecfe46eac745" exitCode=0 Sep 17 15:56:04.646403 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.646337 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" event={"ID":"02a1199f6936144f7915bb63aaf5c379","Type":"ContainerDied","Data":"8ca74c77c401b080f59d3ba3bc29b297bd6716353770b11a1917ecfe46eac745"} Sep 17 15:56:04.647880 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.647859 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vlcft" event={"ID":"edf5c407-8316-462e-b452-f867f1f5f301","Type":"ContainerStarted","Data":"70c4f7336f7a35bfc0983d748bfb29f1d179664f7c0679910402a68c368f6c1c"} Sep 17 15:56:04.651040 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.651011 2580 generic.go:356] "Generic (PLEG): container finished" podID="87ffca99-52b2-48df-bd31-35847f0524e6" containerID="9aea683e2feaebf85415f4f50798340acdbb35ba253b09e67e21dd2aacabb4c0" exitCode=0 Sep 17 15:56:04.651144 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.651061 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdkh6" event={"ID":"87ffca99-52b2-48df-bd31-35847f0524e6","Type":"ContainerDied","Data":"9aea683e2feaebf85415f4f50798340acdbb35ba253b09e67e21dd2aacabb4c0"} Sep 17 15:56:04.658576 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.658527 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-222.ec2.internal" podStartSLOduration=20.658507119 podStartE2EDuration="20.658507119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:03.661324379 +0000 UTC m=+20.764569761" watchObservedRunningTime="2026-09-17 15:56:04.658507119 +0000 UTC m=+21.761752502" Sep 17 15:56:04.658748 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.658726 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2slv4" podStartSLOduration=2.7066100730000002 podStartE2EDuration="21.658719987s" totalImagesPullingTime="18.952109914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:04.658355445 +0000 UTC m=+21.761600829" watchObservedRunningTime="2026-09-17 15:56:04.658719987 +0000 UTC m=+21.761965368" Sep 17 15:56:04.699034 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.698972 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8sb8r" podStartSLOduration=3.32011434 podStartE2EDuration="21.698953315s" totalImagesPullingTime="18.378838975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:04.698471996 +0000 UTC m=+21.801717384" watchObservedRunningTime="2026-09-17 15:56:04.698953315 +0000 UTC m=+21.802198698" Sep 17 15:56:04.750294 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.748003 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vlcft" podStartSLOduration=3.373469407 podStartE2EDuration="21.747980068s" totalImagesPullingTime="18.374510661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:04.747942772 +0000 UTC m=+21.851188155" watchObservedRunningTime="2026-09-17 15:56:04.747980068 +0000 UTC m=+21.851225450" Sep 17 15:56:04.962083 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:04.962060 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 17 15:56:05.292580 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:05.292269 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-17T15:56:04.962076902Z","UUID":"c41b44cf-4464-4706-b60b-f9317f779478","Handler":null,"Name":"","Endpoint":""} Sep 17 15:56:05.295831 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:05.295801 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 17 15:56:05.295972 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:05.295841 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 17 15:56:05.487829 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:05.487760 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:05.488005 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:05.487946 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:05.655786 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:05.655696 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bvqtv" event={"ID":"6039d184-66c2-4e53-b104-6fa1b4717dff","Type":"ContainerStarted","Data":"a1ef3c26909131f89ce126c4cf896715a09e57c9113c716c6e4368facec3e746"} Sep 17 15:56:05.657902 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:05.657875 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" event={"ID":"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d","Type":"ContainerStarted","Data":"15f9e0d6e0e024432927d53a1b32c7906b087a2b76e2f4fd57232ebe2d393eee"} Sep 17 15:56:05.659962 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:05.659927 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" event={"ID":"02a1199f6936144f7915bb63aaf5c379","Type":"ContainerStarted","Data":"786c44a3a0141423b182480ddf30348e700a16a8c6994340e813f528eebb888a"} Sep 17 15:56:05.675438 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:05.675388 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bvqtv" podStartSLOduration=4.279003493 podStartE2EDuration="22.675367819s" totalImagesPullingTime="18.396364326s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:05.675168358 +0000 UTC m=+22.778413740" watchObservedRunningTime="2026-09-17 15:56:05.675367819 +0000 UTC m=+22.778613204" Sep 17 15:56:05.695970 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:05.695914 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-222.ec2.internal" podStartSLOduration=21.695894011 podStartE2EDuration="21.695894011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:55:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:05.695646612 +0000 UTC m=+22.798891994" watchObservedRunningTime="2026-09-17 15:56:05.695894011 +0000 UTC m=+22.799139394" Sep 17 15:56:06.487776 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:06.487738 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:06.487944 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:06.487749 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:06.487944 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:06.487871 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:06.488051 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:06.487939 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:06.666644 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:06.666618 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 15:56:06.667122 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:06.667098 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" event={"ID":"e7d7c2f1-e491-428d-98e5-573bb6cc97d0","Type":"ContainerStarted","Data":"28aa9c78421fd61672c7f9d84d79c0fcce8e9a4d2a4a382db3ecea22c9de10cc"} Sep 17 15:56:06.669633 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:06.669506 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" event={"ID":"25c06c3b-5780-45c9-8a82-c4cfd9d6a57d","Type":"ContainerStarted","Data":"e34c5d27bd9a4c52520cc15c9a85ccbcf24aebc1b2c60b02ccd000b8972f1f94"} Sep 17 15:56:06.689920 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:06.689860 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8v7dl" podStartSLOduration=3.516399894 podStartE2EDuration="23.68984102s" totalImagesPullingTime="20.173441126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:55:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:06.689377027 +0000 UTC m=+23.792622411" watchObservedRunningTime="2026-09-17 15:56:06.68984102 +0000 UTC m=+23.793086406" Sep 17 15:56:07.491859 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:07.491826 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:07.492117 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:07.491949 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:08.080115 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:08.080075 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8sb8r" Sep 17 15:56:08.080940 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:08.080905 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8sb8r" Sep 17 15:56:08.488170 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:08.488140 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:08.488170 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:08.488159 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:08.488427 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:08.488264 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:08.488427 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:08.488391 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:09.258456 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:09.258274 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8sb8r" Sep 17 15:56:09.259284 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:09.258849 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8sb8r" Sep 17 15:56:09.487638 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:09.487603 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:09.487826 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:09.487724 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:09.677642 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:09.677606 2580 generic.go:356] "Generic (PLEG): container finished" podID="87ffca99-52b2-48df-bd31-35847f0524e6" containerID="ed29d980fbd4cdae65bb8b6240a558bfdda272f641b21dd4e7cd99846bc1dc18" exitCode=0 Sep 17 15:56:09.677842 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:09.677678 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdkh6" event={"ID":"87ffca99-52b2-48df-bd31-35847f0524e6","Type":"ContainerDied","Data":"ed29d980fbd4cdae65bb8b6240a558bfdda272f641b21dd4e7cd99846bc1dc18"} Sep 17 15:56:09.681347 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:09.681330 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 15:56:09.681720 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:09.681694 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" event={"ID":"e7d7c2f1-e491-428d-98e5-573bb6cc97d0","Type":"ContainerStarted","Data":"7c799487fe871328c0a429e5502aa7e24a1caca0a8252f0053b6e3db98eea246"} Sep 17 15:56:09.682148 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:09.682133 2580 scope.go:117] "RemoveContainer" containerID="60b25efab516490040ea3c3397d4c702b234c132a49b0834fa2c8ec6f4b9db82" Sep 17 15:56:10.487870 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:10.487836 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:10.488372 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:10.487946 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:10.488372 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:10.487992 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:10.488372 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:10.488093 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:10.687000 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:10.686975 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 15:56:10.687378 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:10.687358 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" event={"ID":"e7d7c2f1-e491-428d-98e5-573bb6cc97d0","Type":"ContainerStarted","Data":"dacff47fe3cd0436be715c42ce6c7f03bd3003c1a3526589054a92b83cb697a6"} Sep 17 15:56:10.687764 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:10.687748 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:56:10.687860 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:10.687804 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:56:10.687860 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:10.687834 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:56:10.702526 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:10.702503 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:56:10.702669 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:10.702639 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:56:10.721568 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:10.721528 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" podStartSLOduration=9.246206064 podStartE2EDuration="27.721513145s" totalImagesPullingTime="18.475307081s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:10.72110631 +0000 UTC m=+27.824351717" watchObservedRunningTime="2026-09-17 15:56:10.721513145 +0000 UTC m=+27.824758528" Sep 17 15:56:11.487997 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:11.487967 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:11.488492 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:11.488060 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:11.693888 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:11.693854 2580 generic.go:356] "Generic (PLEG): container finished" podID="87ffca99-52b2-48df-bd31-35847f0524e6" containerID="2454fbcb71caa296ef879aef7ec27a07f6d1f2b6558f8cf28baa66980ec189b2" exitCode=0 Sep 17 15:56:11.694030 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:11.693944 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdkh6" event={"ID":"87ffca99-52b2-48df-bd31-35847f0524e6","Type":"ContainerDied","Data":"2454fbcb71caa296ef879aef7ec27a07f6d1f2b6558f8cf28baa66980ec189b2"} Sep 17 15:56:12.487720 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:12.487683 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:12.487981 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:12.487684 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:12.487981 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:12.487816 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:12.487981 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:12.487906 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:13.488718 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:13.488687 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:13.489183 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:13.488770 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:13.701157 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:13.701112 2580 generic.go:356] "Generic (PLEG): container finished" podID="87ffca99-52b2-48df-bd31-35847f0524e6" containerID="1171a2cbad26b68cac84849b6bdecee1c83681740b79c45b52df019fcff32d8e" exitCode=0 Sep 17 15:56:13.701351 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:13.701185 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdkh6" event={"ID":"87ffca99-52b2-48df-bd31-35847f0524e6","Type":"ContainerDied","Data":"1171a2cbad26b68cac84849b6bdecee1c83681740b79c45b52df019fcff32d8e"} Sep 17 15:56:14.488187 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:14.488152 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:14.488389 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:14.488302 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:14.488389 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:14.488329 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:14.488499 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:14.488460 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:15.488458 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:15.488427 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:15.488915 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:15.488580 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:16.079876 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:16.079843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:16.080111 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:16.079887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:16.080111 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:16.079986 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:56:16.080111 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:16.079987 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 15:56:16.080111 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:16.080038 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs podName:05ed94c8-e908-40bc-abf2-b770ebc1344a nodeName:}" failed. No retries permitted until 2026-09-17 15:56:48.080025113 +0000 UTC m=+65.183270475 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs") pod "network-metrics-daemon-8clfl" (UID: "05ed94c8-e908-40bc-abf2-b770ebc1344a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:56:16.080111 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:16.080051 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret podName:609285b3-0b12-4a2c-a723-afb1861e75a4 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:48.080045784 +0000 UTC m=+65.183291144 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret") pod "global-pull-secret-syncer-8877v" (UID: "609285b3-0b12-4a2c-a723-afb1861e75a4") : object "kube-system"/"original-pull-secret" not registered Sep 17 15:56:16.180739 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:16.180698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdfhg\" (UniqueName: \"kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg\") pod \"network-check-target-kkn7c\" (UID: \"af1e77ed-0bbf-419b-8a67-029c77b67873\") " pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:16.180928 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:16.180877 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 15:56:16.180928 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:16.180907 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 15:56:16.180928 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:16.180920 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vdfhg for pod openshift-network-diagnostics/network-check-target-kkn7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:56:16.181058 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:16.180970 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg podName:af1e77ed-0bbf-419b-8a67-029c77b67873 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:48.180957234 +0000 UTC m=+65.284202593 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdfhg" (UniqueName: "kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg") pod "network-check-target-kkn7c" (UID: "af1e77ed-0bbf-419b-8a67-029c77b67873") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:56:16.487488 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:16.487414 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:16.487488 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:16.487446 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:16.487719 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:16.487530 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:16.487719 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:16.487676 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:17.487921 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:17.487891 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:17.488422 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:17.488029 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:17.717508 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:17.717474 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdkh6" event={"ID":"87ffca99-52b2-48df-bd31-35847f0524e6","Type":"ContainerStarted","Data":"069c27b1ac68dc81a28e89d1c373bf86d26555cab0a1118c9327c9cbbad26d4d"} Sep 17 15:56:18.487705 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:18.487665 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:18.487891 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:18.487664 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:18.487891 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:18.487785 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:18.487891 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:18.487843 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:18.721667 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:18.721634 2580 generic.go:356] "Generic (PLEG): container finished" podID="87ffca99-52b2-48df-bd31-35847f0524e6" containerID="069c27b1ac68dc81a28e89d1c373bf86d26555cab0a1118c9327c9cbbad26d4d" exitCode=0 Sep 17 15:56:18.721821 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:18.721689 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdkh6" event={"ID":"87ffca99-52b2-48df-bd31-35847f0524e6","Type":"ContainerDied","Data":"069c27b1ac68dc81a28e89d1c373bf86d26555cab0a1118c9327c9cbbad26d4d"} Sep 17 15:56:19.487809 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:19.487775 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:19.487984 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:19.487885 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:19.729355 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:19.729320 2580 generic.go:356] "Generic (PLEG): container finished" podID="87ffca99-52b2-48df-bd31-35847f0524e6" containerID="eb669873aed970dab251af0d16eb0f3f1fb065a7259cca893e69bd6c5d9bde49" exitCode=0 Sep 17 15:56:19.729505 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:19.729368 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdkh6" event={"ID":"87ffca99-52b2-48df-bd31-35847f0524e6","Type":"ContainerDied","Data":"eb669873aed970dab251af0d16eb0f3f1fb065a7259cca893e69bd6c5d9bde49"} Sep 17 15:56:20.488153 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:20.488100 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:20.488557 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:20.488108 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:20.488557 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:20.488230 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:20.488557 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:20.488310 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:20.734312 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:20.734103 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-sdkh6" event={"ID":"87ffca99-52b2-48df-bd31-35847f0524e6","Type":"ContainerStarted","Data":"b8a3e4b2f6358d3a268a00dd738612b6d39c5e8d353340415995c6cb137495e0"} Sep 17 15:56:21.487884 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:21.487855 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:21.488046 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:21.487951 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:22.487785 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:22.487746 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:22.488172 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:22.487790 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:22.488172 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:22.487862 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:22.488172 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:22.487968 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:23.489484 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:23.489448 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:23.489882 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:23.489561 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:24.487780 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:24.487754 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:24.487947 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:24.487756 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:24.487947 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:24.487848 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:24.488016 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:24.487939 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:25.488365 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:25.488336 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:25.488749 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:25.488428 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:26.488014 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:26.487985 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:26.488159 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:26.487985 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:26.488159 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:26.488071 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:26.488159 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:26.488148 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:27.488304 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:27.488269 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:27.488671 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:27.488371 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:28.487747 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:28.487713 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:28.487889 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:28.487805 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:28.487889 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:28.487860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:28.487972 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:28.487948 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:29.488434 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:29.488399 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:29.488804 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:29.488522 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:30.487846 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:30.487816 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:30.488027 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:30.487818 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:30.488027 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:30.487901 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:30.488027 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:30.488014 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:31.488184 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:31.488158 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:31.488557 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:31.488268 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:32.487653 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:32.487628 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:32.487920 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:32.487627 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:32.487920 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:32.487744 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:32.487920 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:32.487796 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:33.488562 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:33.488531 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:33.488983 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:33.488607 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:34.487935 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:34.487900 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:34.488133 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:34.487904 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:34.488133 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:34.488004 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:34.488133 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:34.488091 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:35.487708 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:35.487676 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:35.488077 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:35.487795 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:36.488065 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:36.488029 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:36.488463 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:36.488029 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:36.488463 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:36.488131 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:36.488463 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:36.488199 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:37.487944 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:37.487913 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:37.488111 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:37.488011 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:38.488424 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:38.488393 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:38.488955 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:38.488395 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:38.488955 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:38.488479 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:38.488955 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:38.488603 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:39.487571 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:39.487543 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:39.487740 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:39.487639 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:40.488233 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:40.488192 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:40.488620 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:40.488193 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:40.488620 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:40.488298 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:40.488620 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:40.488365 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:41.487784 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:41.487747 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:41.487961 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:41.487880 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:42.487929 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:42.487898 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:42.487929 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:42.487933 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:42.488424 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:42.488008 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:42.488424 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:42.488124 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:42.706151 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:42.706117 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" podUID="e7d7c2f1-e491-428d-98e5-573bb6cc97d0" containerName="ovnkube-controller" probeResult="failure" output="" Sep 17 15:56:43.488671 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:43.488646 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:43.489148 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:43.488720 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:44.487434 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:44.487405 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:44.487600 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:44.487407 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:44.487600 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:44.487490 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:44.487694 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:44.487595 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:45.488292 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:45.488266 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:45.488684 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:45.488357 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:46.488133 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:46.488106 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:46.488323 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:46.488106 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:46.488323 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:46.488206 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:46.488323 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:46.488278 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:47.487760 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:47.487730 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:47.487950 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:47.487819 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:48.105153 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:48.105123 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:48.105641 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:48.105175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:48.105641 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.105293 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 17 15:56:48.105641 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.105348 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret podName:609285b3-0b12-4a2c-a723-afb1861e75a4 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:52.105334091 +0000 UTC m=+129.208579463 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret") pod "global-pull-secret-syncer-8877v" (UID: "609285b3-0b12-4a2c-a723-afb1861e75a4") : object "kube-system"/"original-pull-secret" not registered Sep 17 15:56:48.105641 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.105294 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:56:48.105641 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.105427 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs podName:05ed94c8-e908-40bc-abf2-b770ebc1344a nodeName:}" failed. No retries permitted until 2026-09-17 15:57:52.105412701 +0000 UTC m=+129.208658070 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs") pod "network-metrics-daemon-8clfl" (UID: "05ed94c8-e908-40bc-abf2-b770ebc1344a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 17 15:56:48.205624 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:48.205598 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdfhg\" (UniqueName: \"kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg\") pod \"network-check-target-kkn7c\" (UID: \"af1e77ed-0bbf-419b-8a67-029c77b67873\") " pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:48.205785 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.205722 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 17 15:56:48.205785 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.205744 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 17 15:56:48.205785 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.205756 2580 projected.go:196] Error preparing data for projected volume kube-api-access-vdfhg for pod openshift-network-diagnostics/network-check-target-kkn7c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:56:48.205937 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.205807 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg podName:af1e77ed-0bbf-419b-8a67-029c77b67873 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:52.205791242 +0000 UTC m=+129.309036603 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdfhg" (UniqueName: "kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg") pod "network-check-target-kkn7c" (UID: "af1e77ed-0bbf-419b-8a67-029c77b67873") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 17 15:56:48.488179 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:48.488110 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:48.488179 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:48.488129 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:48.488388 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.488242 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:48.488388 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.488308 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:48.872339 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:48.872277 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-sdkh6" podStartSLOduration=37.042248333 podStartE2EDuration="1m5.872252602s" totalImagesPullingTime="28.830004269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:55:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:20.773295278 +0000 UTC m=+37.876540657" watchObservedRunningTime="2026-09-17 15:56:48.872252602 +0000 UTC m=+65.975497985" Sep 17 15:56:48.872998 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:48.872976 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kkn7c"] Sep 17 15:56:48.873098 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:48.873077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:48.873171 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.873155 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:48.876850 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:48.876829 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8clfl"] Sep 17 15:56:48.876946 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:48.876906 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:48.876998 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.876981 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:48.891855 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:48.891832 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8877v"] Sep 17 15:56:48.891955 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:48.891899 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:48.891996 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:48.891963 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:49.796488 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:49.796324 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 15:56:49.796819 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:49.796504 2580 generic.go:356] "Generic (PLEG): container finished" podID="cb4be840-5ccd-4ea0-900c-ee3ad4111e15" containerID="9bda7f1dbcbd68cf39500c40a3955ae9bfd760cde40f603502b1d79deea19684" exitCode=1 Sep 17 15:56:49.796819 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:49.796566 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2slv4" event={"ID":"cb4be840-5ccd-4ea0-900c-ee3ad4111e15","Type":"ContainerDied","Data":"9bda7f1dbcbd68cf39500c40a3955ae9bfd760cde40f603502b1d79deea19684"} Sep 17 15:56:49.796904 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:49.796889 2580 scope.go:117] "RemoveContainer" containerID="9bda7f1dbcbd68cf39500c40a3955ae9bfd760cde40f603502b1d79deea19684" Sep 17 15:56:50.487482 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:50.487407 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:50.487619 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:50.487413 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:50.487619 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:50.487516 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:50.487619 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:50.487422 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:50.487778 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:50.487583 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:50.487778 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:50.487701 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:50.800018 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:50.799995 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 15:56:50.800587 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:50.800059 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2slv4" event={"ID":"cb4be840-5ccd-4ea0-900c-ee3ad4111e15","Type":"ContainerStarted","Data":"9c6954d1e4f8b5c92dd93a2125bdf273d61ab6d56b50b898c376585ba6d5a27c"} Sep 17 15:56:52.487970 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.487899 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:52.488339 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.487905 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:52.488339 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.487987 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8877v" podUID="609285b3-0b12-4a2c-a723-afb1861e75a4" Sep 17 15:56:52.488339 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.487908 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:52.488339 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.488074 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8clfl" podUID="05ed94c8-e908-40bc-abf2-b770ebc1344a" Sep 17 15:56:52.488339 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.488112 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kkn7c" podUID="af1e77ed-0bbf-419b-8a67-029c77b67873" Sep 17 15:56:52.652854 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.652831 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-222.ec2.internal" event="NodeReady" Sep 17 15:56:52.652977 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.652928 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 17 15:56:52.699830 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.699792 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-wps4t"] Sep 17 15:56:52.702102 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.702078 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.702736 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.702717 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms"] Sep 17 15:56:52.704351 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.704337 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f94b94954-h5n4x"] Sep 17 15:56:52.704461 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.704446 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:56:52.705381 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.705363 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 17 15:56:52.705882 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.705866 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.705963 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.705907 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 17 15:56:52.706024 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.705871 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 17 15:56:52.706270 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.706256 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ftgsz\"" Sep 17 15:56:52.707444 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.707426 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 17 15:56:52.707547 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.707455 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 17 15:56:52.707547 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.707485 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 17 15:56:52.709623 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.709605 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 17 15:56:52.710014 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.709996 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-t4tp8\"" Sep 17 15:56:52.711863 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.711844 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 17 15:56:52.711979 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.711849 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 17 15:56:52.711979 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.711955 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 17 15:56:52.712077 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.712047 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g5p99\"" Sep 17 15:56:52.712251 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.712233 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 17 15:56:52.718382 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.718361 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 17 15:56:52.718559 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.718545 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-wps4t"] Sep 17 15:56:52.719271 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.719078 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms"] Sep 17 15:56:52.719973 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.719955 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 17 15:56:52.723760 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.723742 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh"] Sep 17 15:56:52.725762 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.725746 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc"] Sep 17 15:56:52.725872 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.725858 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" Sep 17 15:56:52.727346 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.727331 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r"] Sep 17 15:56:52.727450 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.727434 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" Sep 17 15:56:52.728645 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.728625 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 17 15:56:52.728721 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.728676 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9fmc7\"" Sep 17 15:56:52.728903 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.728890 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 17 15:56:52.729031 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.729016 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-86dfcf9bc6-7j9bp"] Sep 17 15:56:52.729114 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.729101 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:56:52.731246 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.731210 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd"] Sep 17 15:56:52.731304 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.731290 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.733197 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.733179 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-pnkdl"] Sep 17 15:56:52.733316 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.733301 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:56:52.735100 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.735081 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-pnkdl" Sep 17 15:56:52.735352 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.735156 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-88zv4"] Sep 17 15:56:52.736848 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.736685 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 17 15:56:52.737049 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.736872 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 17 15:56:52.737588 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.737290 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 17 15:56:52.738153 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.738100 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 17 15:56:52.738322 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.738300 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.738692 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.740724 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.740778 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.740984 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-88zv4" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741014 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741095 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a11d802-951f-4a33-bb26-7b2d2b641ca6-image-registry-private-configuration\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741136 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741173 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741201 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sscqp\" (UniqueName: \"kubernetes.io/projected/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-kube-api-access-sscqp\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741299 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a11d802-951f-4a33-bb26-7b2d2b641ca6-ca-trust-extracted\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741358 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-snapshots\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741434 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-serving-cert\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741466 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741589 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741657 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741684 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-bound-sa-token\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.742864 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741731 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5tjr\" (UniqueName: \"kubernetes.io/projected/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-kube-api-access-d5tjr\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.743675 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741833 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a11d802-951f-4a33-bb26-7b2d2b641ca6-installation-pull-secrets\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.743675 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-tmp\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.743675 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741879 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-pbk4w\"" Sep 17 15:56:52.743675 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741895 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 17 15:56:52.743675 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741909 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-certificates\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.743675 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67bn9\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-kube-api-access-67bn9\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.743675 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.741976 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a11d802-951f-4a33-bb26-7b2d2b641ca6-trusted-ca\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.743675 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.742023 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r"] Sep 17 15:56:52.743675 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.742165 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 17 15:56:52.743675 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.742395 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rw9sn\"" Sep 17 15:56:52.743675 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.742633 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fq8c5\"" Sep 17 15:56:52.743675 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.742747 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 17 15:56:52.744283 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.744171 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-x2mz4\"" Sep 17 15:56:52.744480 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.744462 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 17 15:56:52.744550 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.744481 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 17 15:56:52.744550 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.744491 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 17 15:56:52.744654 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.744562 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 17 15:56:52.744654 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.744611 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 17 15:56:52.744869 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.744832 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-7tjtr\"" Sep 17 15:56:52.744869 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.744845 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 17 15:56:52.744972 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.744956 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dxj9s\"" Sep 17 15:56:52.745024 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.745004 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 17 15:56:52.745103 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.745083 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 17 15:56:52.746146 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.746130 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh"] Sep 17 15:56:52.746264 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.746250 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r" Sep 17 15:56:52.748257 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.748239 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5lzg5"] Sep 17 15:56:52.748346 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.748293 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.750010 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.749994 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz"] Sep 17 15:56:52.750136 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.750100 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5lzg5" Sep 17 15:56:52.752362 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.752339 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f94b94954-h5n4x"] Sep 17 15:56:52.752502 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.752422 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:56:52.752590 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.752556 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r"] Sep 17 15:56:52.753146 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.753126 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-qp4q9"] Sep 17 15:56:52.753856 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.753835 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Sep 17 15:56:52.755967 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.755931 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-nnrq6\"" Sep 17 15:56:52.756244 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.756209 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 17 15:56:52.756555 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.756533 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh"] Sep 17 15:56:52.756642 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.756571 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc"] Sep 17 15:56:52.756642 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.756592 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-86dfcf9bc6-7j9bp"] Sep 17 15:56:52.756642 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.756538 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 17 15:56:52.756642 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.756618 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.757008 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.756987 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 17 15:56:52.757100 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.757080 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Sep 17 15:56:52.757156 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.757133 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Sep 17 15:56:52.757295 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.757278 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-88zv4"] Sep 17 15:56:52.757367 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.757300 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 17 15:56:52.757417 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.757368 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 17 15:56:52.757475 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.757436 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 17 15:56:52.757525 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.757503 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 17 15:56:52.757604 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.757589 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-lc6hf\"" Sep 17 15:56:52.757695 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.757679 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Sep 17 15:56:52.758406 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.758361 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r"] Sep 17 15:56:52.759106 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.759085 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5lzg5"] Sep 17 15:56:52.759923 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.759824 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz"] Sep 17 15:56:52.760265 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.760148 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 17 15:56:52.760265 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.760196 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 17 15:56:52.760265 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.760237 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-nzdwt\"" Sep 17 15:56:52.761846 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.761830 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 17 15:56:52.762232 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.762194 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 17 15:56:52.767236 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.767197 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-qp4q9"] Sep 17 15:56:52.768276 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.768258 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 17 15:56:52.776102 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.776082 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-pnkdl"] Sep 17 15:56:52.778247 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.778204 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd"] Sep 17 15:56:52.785470 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.785451 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh"] Sep 17 15:56:52.815645 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.815617 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2bdcw"] Sep 17 15:56:52.818323 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.818301 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:52.821319 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.821301 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 17 15:56:52.821505 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.821486 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 17 15:56:52.821572 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.821516 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 17 15:56:52.821572 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.821524 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 17 15:56:52.821940 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.821863 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xrk56\"" Sep 17 15:56:52.831063 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.831046 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2bdcw"] Sep 17 15:56:52.842269 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5tjr\" (UniqueName: \"kubernetes.io/projected/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-kube-api-access-d5tjr\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.842360 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842286 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f94a7e8-2cf1-4051-a1d5-c7d29241fab3-cert\") pod \"ingress-canary-88zv4\" (UID: \"5f94a7e8-2cf1-4051-a1d5-c7d29241fab3\") " pod="openshift-ingress-canary/ingress-canary-88zv4" Sep 17 15:56:52.842360 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842317 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a11d802-951f-4a33-bb26-7b2d2b641ca6-installation-pull-secrets\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.842360 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842339 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d24009de-229f-4bb3-a49e-5499cfc0b519-config-volume\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:52.842455 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842363 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzf6m\" (UniqueName: \"kubernetes.io/projected/d24009de-229f-4bb3-a49e-5499cfc0b519-kube-api-access-hzf6m\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:52.842455 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-9f49r\" (UID: \"7ad114b5-1283-442f-b7b2-fde01ae0f839\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:56:52.842455 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842413 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmv7w\" (UniqueName: \"kubernetes.io/projected/99684b4e-9484-4867-a730-ea1662218945-kube-api-access-tmv7w\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.842455 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67bn9\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-kube-api-access-67bn9\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.842570 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842473 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d24009de-229f-4bb3-a49e-5499cfc0b519-tmp-dir\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:52.842570 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842494 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-stats-auth\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.842570 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a11d802-951f-4a33-bb26-7b2d2b641ca6-trusted-ca\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.842570 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a11d802-951f-4a33-bb26-7b2d2b641ca6-image-registry-private-configuration\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.842685 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842633 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9gw6\" (UniqueName: \"kubernetes.io/projected/4861c6ad-a4a3-4947-a1f1-b21601e7a455-kube-api-access-r9gw6\") pod \"kube-storage-version-migrator-operator-55889f7b47-lh7sh\" (UID: \"4861c6ad-a4a3-4947-a1f1-b21601e7a455\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" Sep 17 15:56:52.842685 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842670 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51b96393-ed39-45b3-baca-b0acb42c254b-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-ghjkd\" (UID: \"51b96393-ed39-45b3-baca-b0acb42c254b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:56:52.842743 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842698 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.842778 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842755 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4861c6ad-a4a3-4947-a1f1-b21601e7a455-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-lh7sh\" (UID: \"4861c6ad-a4a3-4947-a1f1-b21601e7a455\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" Sep 17 15:56:52.842818 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842801 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a11d802-951f-4a33-bb26-7b2d2b641ca6-ca-trust-extracted\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.842852 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7r6w\" (UniqueName: \"kubernetes.io/projected/060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4-kube-api-access-z7r6w\") pod \"service-ca-operator-57cdc4b768-mmrtc\" (UID: \"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" Sep 17 15:56:52.842891 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842861 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/49f537d0-1508-448d-af52-bc6b8fe22ece-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.842946 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842928 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.843008 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842955 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-snapshots\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.843008 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842972 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:52.843008 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.842986 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a3b84c4-ac8d-4e8a-a606-aba79b7fe148-trusted-ca\") pod \"console-operator-fd4bb46c-qp4q9\" (UID: \"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148\") " pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.843129 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f62b1102-06d5-4996-bdcc-f75199fe0464-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-858cb56476-tdh4r\" (UID: \"f62b1102-06d5-4996-bdcc-f75199fe0464\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r" Sep 17 15:56:52.843129 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.843077 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 15:56:52.843129 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.843089 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f94b94954-h5n4x: secret "image-registry-tls" not found Sep 17 15:56:52.843129 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843093 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a11d802-951f-4a33-bb26-7b2d2b641ca6-ca-trust-extracted\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.843315 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.843156 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls podName:0a11d802-951f-4a33-bb26-7b2d2b641ca6 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:53.34313694 +0000 UTC m=+70.446382316 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls") pod "image-registry-6f94b94954-h5n4x" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6") : secret "image-registry-tls" not found Sep 17 15:56:52.843315 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843086 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-serving-cert\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.843315 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843233 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4-serving-cert\") pod \"service-ca-operator-57cdc4b768-mmrtc\" (UID: \"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" Sep 17 15:56:52.843315 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/49f537d0-1508-448d-af52-bc6b8fe22ece-ca\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.843315 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843285 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52fe5bcc-c834-460f-bf01-cc292c18c146-tmp\") pod \"klusterlet-addon-workmgr-5c44cc5f4-s5fxz\" (UID: \"52fe5bcc-c834-460f-bf01-cc292c18c146\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:56:52.843315 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.843567 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843337 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt66w\" (UniqueName: \"kubernetes.io/projected/790edfda-48c6-4852-9b6f-e238976373be-kube-api-access-wt66w\") pod \"network-check-source-7cbc45fddc-pnkdl\" (UID: \"790edfda-48c6-4852-9b6f-e238976373be\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-pnkdl" Sep 17 15:56:52.843567 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-bound-sa-token\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.843567 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4861c6ad-a4a3-4947-a1f1-b21601e7a455-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-lh7sh\" (UID: \"4861c6ad-a4a3-4947-a1f1-b21601e7a455\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" Sep 17 15:56:52.843567 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843424 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n9zr\" (UniqueName: \"kubernetes.io/projected/f62b1102-06d5-4996-bdcc-f75199fe0464-kube-api-access-8n9zr\") pod \"managed-serviceaccount-addon-agent-858cb56476-tdh4r\" (UID: \"f62b1102-06d5-4996-bdcc-f75199fe0464\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r" Sep 17 15:56:52.843567 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843468 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-tmp\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.843567 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843492 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/49f537d0-1508-448d-af52-bc6b8fe22ece-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.843567 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-certificates\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.843567 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843557 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/49f537d0-1508-448d-af52-bc6b8fe22ece-hub\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.843877 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843581 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r748f\" (UniqueName: \"kubernetes.io/projected/49f537d0-1508-448d-af52-bc6b8fe22ece-kube-api-access-r748f\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.843877 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843585 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a11d802-951f-4a33-bb26-7b2d2b641ca6-trusted-ca\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.843877 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843604 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a3b84c4-ac8d-4e8a-a606-aba79b7fe148-config\") pod \"console-operator-fd4bb46c-qp4q9\" (UID: \"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148\") " pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.843877 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843644 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxdjn\" (UniqueName: \"kubernetes.io/projected/7ad114b5-1283-442f-b7b2-fde01ae0f839-kube-api-access-mxdjn\") pod \"cluster-samples-operator-685d7d68b5-9f49r\" (UID: \"7ad114b5-1283-442f-b7b2-fde01ae0f839\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:56:52.843877 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843661 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-snapshots\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.843877 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843672 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-default-certificate\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.843877 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.844154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.844154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843915 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:56:52.844154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sscqp\" (UniqueName: \"kubernetes.io/projected/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-kube-api-access-sscqp\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:56:52.844154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843971 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-tmp\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.844154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.843977 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k42k\" (UniqueName: \"kubernetes.io/projected/0a3b84c4-ac8d-4e8a-a606-aba79b7fe148-kube-api-access-2k42k\") pod \"console-operator-fd4bb46c-qp4q9\" (UID: \"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148\") " pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.844154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4mzt\" (UniqueName: \"kubernetes.io/projected/52fe5bcc-c834-460f-bf01-cc292c18c146-kube-api-access-f4mzt\") pod \"klusterlet-addon-workmgr-5c44cc5f4-s5fxz\" (UID: \"52fe5bcc-c834-460f-bf01-cc292c18c146\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:56:52.844154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844082 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xnn7\" (UniqueName: \"kubernetes.io/projected/15601992-7dfc-4244-848a-eccecf6bc1c5-kube-api-access-7xnn7\") pod \"volume-data-source-validator-5c4db8bdf9-5lzg5\" (UID: \"15601992-7dfc-4244-848a-eccecf6bc1c5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5lzg5" Sep 17 15:56:52.844154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844122 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-certificates\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.844154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844133 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/52fe5bcc-c834-460f-bf01-cc292c18c146-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c44cc5f4-s5fxz\" (UID: \"52fe5bcc-c834-460f-bf01-cc292c18c146\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:56:52.844563 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844172 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.844563 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844184 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ghjkd\" (UID: \"51b96393-ed39-45b3-baca-b0acb42c254b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:56:52.844563 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844236 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:56:52.844563 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4-config\") pod \"service-ca-operator-57cdc4b768-mmrtc\" (UID: \"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" Sep 17 15:56:52.844563 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.844312 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 15:56:52.844563 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844349 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lt62\" (UniqueName: \"kubernetes.io/projected/5f94a7e8-2cf1-4051-a1d5-c7d29241fab3-kube-api-access-4lt62\") pod \"ingress-canary-88zv4\" (UID: \"5f94a7e8-2cf1-4051-a1d5-c7d29241fab3\") " pod="openshift-ingress-canary/ingress-canary-88zv4" Sep 17 15:56:52.844563 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.844413 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls podName:9ac10347-81ed-4b23-8eac-ddc2a4c1d171 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:53.344396394 +0000 UTC m=+70.447641762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-5fpms" (UID: "9ac10347-81ed-4b23-8eac-ddc2a4c1d171") : secret "cluster-monitoring-operator-tls" not found Sep 17 15:56:52.844563 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844474 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/49f537d0-1508-448d-af52-bc6b8fe22ece-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.844563 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a3b84c4-ac8d-4e8a-a606-aba79b7fe148-serving-cert\") pod \"console-operator-fd4bb46c-qp4q9\" (UID: \"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148\") " pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.844975 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844569 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.844975 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.844767 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-telemetry-config\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:56:52.846633 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.846614 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-serving-cert\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.846740 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.846720 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a11d802-951f-4a33-bb26-7b2d2b641ca6-installation-pull-secrets\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.846786 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.846760 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a11d802-951f-4a33-bb26-7b2d2b641ca6-image-registry-private-configuration\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.868177 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.868149 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67bn9\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-kube-api-access-67bn9\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.868843 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.868825 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-bound-sa-token\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:52.869097 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.869077 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sscqp\" (UniqueName: \"kubernetes.io/projected/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-kube-api-access-sscqp\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:56:52.870248 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.870208 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5tjr\" (UniqueName: \"kubernetes.io/projected/fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b-kube-api-access-d5tjr\") pod \"insights-operator-6f9c644dd9-wps4t\" (UID: \"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b\") " pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:52.927580 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.927547 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:56:52.945601 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.945574 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a3b84c4-ac8d-4e8a-a606-aba79b7fe148-trusted-ca\") pod \"console-operator-fd4bb46c-qp4q9\" (UID: \"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148\") " pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.945727 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.945605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f62b1102-06d5-4996-bdcc-f75199fe0464-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-858cb56476-tdh4r\" (UID: \"f62b1102-06d5-4996-bdcc-f75199fe0464\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r" Sep 17 15:56:52.945727 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.945624 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4-serving-cert\") pod \"service-ca-operator-57cdc4b768-mmrtc\" (UID: \"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" Sep 17 15:56:52.945727 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.945640 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/49f537d0-1508-448d-af52-bc6b8fe22ece-ca\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.945727 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.945693 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52fe5bcc-c834-460f-bf01-cc292c18c146-tmp\") pod \"klusterlet-addon-workmgr-5c44cc5f4-s5fxz\" (UID: \"52fe5bcc-c834-460f-bf01-cc292c18c146\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:56:52.945727 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.945724 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt66w\" (UniqueName: \"kubernetes.io/projected/790edfda-48c6-4852-9b6f-e238976373be-kube-api-access-wt66w\") pod \"network-check-source-7cbc45fddc-pnkdl\" (UID: \"790edfda-48c6-4852-9b6f-e238976373be\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-pnkdl" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.945747 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4861c6ad-a4a3-4947-a1f1-b21601e7a455-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-lh7sh\" (UID: \"4861c6ad-a4a3-4947-a1f1-b21601e7a455\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.945771 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n9zr\" (UniqueName: \"kubernetes.io/projected/f62b1102-06d5-4996-bdcc-f75199fe0464-kube-api-access-8n9zr\") pod \"managed-serviceaccount-addon-agent-858cb56476-tdh4r\" (UID: \"f62b1102-06d5-4996-bdcc-f75199fe0464\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.945889 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/49f537d0-1508-448d-af52-bc6b8fe22ece-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.945925 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/49f537d0-1508-448d-af52-bc6b8fe22ece-hub\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.945952 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r748f\" (UniqueName: \"kubernetes.io/projected/49f537d0-1508-448d-af52-bc6b8fe22ece-kube-api-access-r748f\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.945977 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a3b84c4-ac8d-4e8a-a606-aba79b7fe148-config\") pod \"console-operator-fd4bb46c-qp4q9\" (UID: \"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148\") " pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946007 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxdjn\" (UniqueName: \"kubernetes.io/projected/7ad114b5-1283-442f-b7b2-fde01ae0f839-kube-api-access-mxdjn\") pod \"cluster-samples-operator-685d7d68b5-9f49r\" (UID: \"7ad114b5-1283-442f-b7b2-fde01ae0f839\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946033 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-default-certificate\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946086 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k42k\" (UniqueName: \"kubernetes.io/projected/0a3b84c4-ac8d-4e8a-a606-aba79b7fe148-kube-api-access-2k42k\") pod \"console-operator-fd4bb46c-qp4q9\" (UID: \"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148\") " pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946100 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52fe5bcc-c834-460f-bf01-cc292c18c146-tmp\") pod \"klusterlet-addon-workmgr-5c44cc5f4-s5fxz\" (UID: \"52fe5bcc-c834-460f-bf01-cc292c18c146\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946112 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4mzt\" (UniqueName: \"kubernetes.io/projected/52fe5bcc-c834-460f-bf01-cc292c18c146-kube-api-access-f4mzt\") pod \"klusterlet-addon-workmgr-5c44cc5f4-s5fxz\" (UID: \"52fe5bcc-c834-460f-bf01-cc292c18c146\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xnn7\" (UniqueName: \"kubernetes.io/projected/15601992-7dfc-4244-848a-eccecf6bc1c5-kube-api-access-7xnn7\") pod \"volume-data-source-validator-5c4db8bdf9-5lzg5\" (UID: \"15601992-7dfc-4244-848a-eccecf6bc1c5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5lzg5" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/52fe5bcc-c834-460f-bf01-cc292c18c146-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c44cc5f4-s5fxz\" (UID: \"52fe5bcc-c834-460f-bf01-cc292c18c146\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:56:52.946471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946467 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ghjkd\" (UID: \"51b96393-ed39-45b3-baca-b0acb42c254b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946505 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4-config\") pod \"service-ca-operator-57cdc4b768-mmrtc\" (UID: \"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946536 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lt62\" (UniqueName: \"kubernetes.io/projected/5f94a7e8-2cf1-4051-a1d5-c7d29241fab3-kube-api-access-4lt62\") pod \"ingress-canary-88zv4\" (UID: \"5f94a7e8-2cf1-4051-a1d5-c7d29241fab3\") " pod="openshift-ingress-canary/ingress-canary-88zv4" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946571 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/49f537d0-1508-448d-af52-bc6b8fe22ece-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946585 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a3b84c4-ac8d-4e8a-a606-aba79b7fe148-trusted-ca\") pod \"console-operator-fd4bb46c-qp4q9\" (UID: \"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148\") " pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a3b84c4-ac8d-4e8a-a606-aba79b7fe148-serving-cert\") pod \"console-operator-fd4bb46c-qp4q9\" (UID: \"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148\") " pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f94a7e8-2cf1-4051-a1d5-c7d29241fab3-cert\") pod \"ingress-canary-88zv4\" (UID: \"5f94a7e8-2cf1-4051-a1d5-c7d29241fab3\") " pod="openshift-ingress-canary/ingress-canary-88zv4" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.946988 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d24009de-229f-4bb3-a49e-5499cfc0b519-config-volume\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947023 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzf6m\" (UniqueName: \"kubernetes.io/projected/d24009de-229f-4bb3-a49e-5499cfc0b519-kube-api-access-hzf6m\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-9f49r\" (UID: \"7ad114b5-1283-442f-b7b2-fde01ae0f839\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947078 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmv7w\" (UniqueName: \"kubernetes.io/projected/99684b4e-9484-4867-a730-ea1662218945-kube-api-access-tmv7w\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947110 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d24009de-229f-4bb3-a49e-5499cfc0b519-tmp-dir\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-stats-auth\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.947185 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9gw6\" (UniqueName: \"kubernetes.io/projected/4861c6ad-a4a3-4947-a1f1-b21601e7a455-kube-api-access-r9gw6\") pod \"kube-storage-version-migrator-operator-55889f7b47-lh7sh\" (UID: \"4861c6ad-a4a3-4947-a1f1-b21601e7a455\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" Sep 17 15:56:52.947825 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947232 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51b96393-ed39-45b3-baca-b0acb42c254b-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-ghjkd\" (UID: \"51b96393-ed39-45b3-baca-b0acb42c254b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:56:52.947825 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947260 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.947825 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947284 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4861c6ad-a4a3-4947-a1f1-b21601e7a455-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-lh7sh\" (UID: \"4861c6ad-a4a3-4947-a1f1-b21601e7a455\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" Sep 17 15:56:52.947825 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947320 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7r6w\" (UniqueName: \"kubernetes.io/projected/060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4-kube-api-access-z7r6w\") pod \"service-ca-operator-57cdc4b768-mmrtc\" (UID: \"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" Sep 17 15:56:52.947825 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/49f537d0-1508-448d-af52-bc6b8fe22ece-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.947825 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947389 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:52.947825 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.947504 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 15:56:52.947825 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.947506 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4-config\") pod \"service-ca-operator-57cdc4b768-mmrtc\" (UID: \"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" Sep 17 15:56:52.947825 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.947564 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls podName:d24009de-229f-4bb3-a49e-5499cfc0b519 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:53.447547495 +0000 UTC m=+70.550792857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls") pod "dns-default-2bdcw" (UID: "d24009de-229f-4bb3-a49e-5499cfc0b519") : secret "dns-default-metrics-tls" not found Sep 17 15:56:52.947825 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.947567 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 15:56:52.947825 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.947609 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert podName:51b96393-ed39-45b3-baca-b0acb42c254b nodeName:}" failed. No retries permitted until 2026-09-17 15:56:53.44759821 +0000 UTC m=+70.550843586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-ghjkd" (UID: "51b96393-ed39-45b3-baca-b0acb42c254b") : secret "networking-console-plugin-cert" not found Sep 17 15:56:52.948364 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.948281 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a3b84c4-ac8d-4e8a-a606-aba79b7fe148-config\") pod \"console-operator-fd4bb46c-qp4q9\" (UID: \"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148\") " pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.948419 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.948363 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 15:56:52.948419 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.948412 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs podName:99684b4e-9484-4867-a730-ea1662218945 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:53.448396837 +0000 UTC m=+70.551642199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs") pod "router-default-86dfcf9bc6-7j9bp" (UID: "99684b4e-9484-4867-a730-ea1662218945") : secret "router-metrics-certs-default" not found Sep 17 15:56:52.948980 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.948899 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/49f537d0-1508-448d-af52-bc6b8fe22ece-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.949184 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.949159 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 15:56:52.949299 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.949239 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls podName:7ad114b5-1283-442f-b7b2-fde01ae0f839 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:53.449198505 +0000 UTC m=+70.552443879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-9f49r" (UID: "7ad114b5-1283-442f-b7b2-fde01ae0f839") : secret "samples-operator-tls" not found Sep 17 15:56:52.949566 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.949514 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/49f537d0-1508-448d-af52-bc6b8fe22ece-hub\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.949745 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:52.949675 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle podName:99684b4e-9484-4867-a730-ea1662218945 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:53.449659534 +0000 UTC m=+70.552904909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle") pod "router-default-86dfcf9bc6-7j9bp" (UID: "99684b4e-9484-4867-a730-ea1662218945") : configmap references non-existent config key: service-ca.crt Sep 17 15:56:52.949838 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.949761 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d24009de-229f-4bb3-a49e-5499cfc0b519-tmp-dir\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:52.949838 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.949780 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d24009de-229f-4bb3-a49e-5499cfc0b519-config-volume\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:52.950259 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.950191 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4-serving-cert\") pod \"service-ca-operator-57cdc4b768-mmrtc\" (UID: \"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" Sep 17 15:56:52.950359 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.950265 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4861c6ad-a4a3-4947-a1f1-b21601e7a455-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-lh7sh\" (UID: \"4861c6ad-a4a3-4947-a1f1-b21601e7a455\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" Sep 17 15:56:52.950431 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.950360 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4861c6ad-a4a3-4947-a1f1-b21601e7a455-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-lh7sh\" (UID: \"4861c6ad-a4a3-4947-a1f1-b21601e7a455\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" Sep 17 15:56:52.950488 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.950471 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51b96393-ed39-45b3-baca-b0acb42c254b-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-ghjkd\" (UID: \"51b96393-ed39-45b3-baca-b0acb42c254b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:56:52.950743 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.950720 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-default-certificate\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.950881 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.950855 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f62b1102-06d5-4996-bdcc-f75199fe0464-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-858cb56476-tdh4r\" (UID: \"f62b1102-06d5-4996-bdcc-f75199fe0464\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r" Sep 17 15:56:52.951438 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.951416 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a3b84c4-ac8d-4e8a-a606-aba79b7fe148-serving-cert\") pod \"console-operator-fd4bb46c-qp4q9\" (UID: \"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148\") " pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.952352 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.952328 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/52fe5bcc-c834-460f-bf01-cc292c18c146-klusterlet-config\") pod \"klusterlet-addon-workmgr-5c44cc5f4-s5fxz\" (UID: \"52fe5bcc-c834-460f-bf01-cc292c18c146\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:56:52.952492 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.952472 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/49f537d0-1508-448d-af52-bc6b8fe22ece-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.952560 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.952472 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5f94a7e8-2cf1-4051-a1d5-c7d29241fab3-cert\") pod \"ingress-canary-88zv4\" (UID: \"5f94a7e8-2cf1-4051-a1d5-c7d29241fab3\") " pod="openshift-ingress-canary/ingress-canary-88zv4" Sep 17 15:56:52.952609 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.952561 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/49f537d0-1508-448d-af52-bc6b8fe22ece-ca\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.952609 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.952595 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-stats-auth\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.952843 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.952823 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/49f537d0-1508-448d-af52-bc6b8fe22ece-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.963486 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.963459 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt66w\" (UniqueName: \"kubernetes.io/projected/790edfda-48c6-4852-9b6f-e238976373be-kube-api-access-wt66w\") pod \"network-check-source-7cbc45fddc-pnkdl\" (UID: \"790edfda-48c6-4852-9b6f-e238976373be\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-pnkdl" Sep 17 15:56:52.964296 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.964278 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4mzt\" (UniqueName: \"kubernetes.io/projected/52fe5bcc-c834-460f-bf01-cc292c18c146-kube-api-access-f4mzt\") pod \"klusterlet-addon-workmgr-5c44cc5f4-s5fxz\" (UID: \"52fe5bcc-c834-460f-bf01-cc292c18c146\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:56:52.968134 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.968111 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmv7w\" (UniqueName: \"kubernetes.io/projected/99684b4e-9484-4867-a730-ea1662218945-kube-api-access-tmv7w\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:52.969014 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.968995 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n9zr\" (UniqueName: \"kubernetes.io/projected/f62b1102-06d5-4996-bdcc-f75199fe0464-kube-api-access-8n9zr\") pod \"managed-serviceaccount-addon-agent-858cb56476-tdh4r\" (UID: \"f62b1102-06d5-4996-bdcc-f75199fe0464\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r" Sep 17 15:56:52.970059 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.970039 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k42k\" (UniqueName: \"kubernetes.io/projected/0a3b84c4-ac8d-4e8a-a606-aba79b7fe148-kube-api-access-2k42k\") pod \"console-operator-fd4bb46c-qp4q9\" (UID: \"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148\") " pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:52.970693 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.970666 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lt62\" (UniqueName: \"kubernetes.io/projected/5f94a7e8-2cf1-4051-a1d5-c7d29241fab3-kube-api-access-4lt62\") pod \"ingress-canary-88zv4\" (UID: \"5f94a7e8-2cf1-4051-a1d5-c7d29241fab3\") " pod="openshift-ingress-canary/ingress-canary-88zv4" Sep 17 15:56:52.970850 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.970830 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxdjn\" (UniqueName: \"kubernetes.io/projected/7ad114b5-1283-442f-b7b2-fde01ae0f839-kube-api-access-mxdjn\") pod \"cluster-samples-operator-685d7d68b5-9f49r\" (UID: \"7ad114b5-1283-442f-b7b2-fde01ae0f839\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:56:52.970850 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.970844 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xnn7\" (UniqueName: \"kubernetes.io/projected/15601992-7dfc-4244-848a-eccecf6bc1c5-kube-api-access-7xnn7\") pod \"volume-data-source-validator-5c4db8bdf9-5lzg5\" (UID: \"15601992-7dfc-4244-848a-eccecf6bc1c5\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5lzg5" Sep 17 15:56:52.970929 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.970892 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzf6m\" (UniqueName: \"kubernetes.io/projected/d24009de-229f-4bb3-a49e-5499cfc0b519-kube-api-access-hzf6m\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:52.970929 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.970895 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9gw6\" (UniqueName: \"kubernetes.io/projected/4861c6ad-a4a3-4947-a1f1-b21601e7a455-kube-api-access-r9gw6\") pod \"kube-storage-version-migrator-operator-55889f7b47-lh7sh\" (UID: \"4861c6ad-a4a3-4947-a1f1-b21601e7a455\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" Sep 17 15:56:52.971646 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.971631 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r748f\" (UniqueName: \"kubernetes.io/projected/49f537d0-1508-448d-af52-bc6b8fe22ece-kube-api-access-r748f\") pod \"cluster-proxy-proxy-agent-59f4858d95-kxhhh\" (UID: \"49f537d0-1508-448d-af52-bc6b8fe22ece\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:52.977868 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:52.977849 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7r6w\" (UniqueName: \"kubernetes.io/projected/060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4-kube-api-access-z7r6w\") pod \"service-ca-operator-57cdc4b768-mmrtc\" (UID: \"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4\") " pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" Sep 17 15:56:53.011972 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.011919 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" Sep 17 15:56:53.034961 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.034942 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" Sep 17 15:56:53.044921 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.044897 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" Sep 17 15:56:53.073968 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.073282 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-pnkdl" Sep 17 15:56:53.086151 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.085940 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-88zv4" Sep 17 15:56:53.091816 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.091733 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:56:53.100413 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.099397 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r" Sep 17 15:56:53.116603 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.115828 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:56:53.118648 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.118521 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5lzg5" Sep 17 15:56:53.126877 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.126620 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:56:53.131972 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.131550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:56:53.314145 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.314119 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-wps4t"] Sep 17 15:56:53.318111 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:56:53.318085 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd31c961_38dd_4bc2_b5e7_f6e1cf778a2b.slice/crio-d312f886582552e9852f24a4cab757cdfb5980123edc7b2ac145e6da5d76a99f WatchSource:0}: Error finding container d312f886582552e9852f24a4cab757cdfb5980123edc7b2ac145e6da5d76a99f: Status 404 returned error can't find the container with id d312f886582552e9852f24a4cab757cdfb5980123edc7b2ac145e6da5d76a99f Sep 17 15:56:53.344910 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.344889 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc"] Sep 17 15:56:53.347149 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:56:53.347127 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod060ea9fe_ba22_4ee2_a7dc_a5e6b46ab4d4.slice/crio-9e5c3c2406eef98b2d625dd9b7c54aaa7d1a5af7c8b6242f5b76dddb0f7eea00 WatchSource:0}: Error finding container 9e5c3c2406eef98b2d625dd9b7c54aaa7d1a5af7c8b6242f5b76dddb0f7eea00: Status 404 returned error can't find the container with id 9e5c3c2406eef98b2d625dd9b7c54aaa7d1a5af7c8b6242f5b76dddb0f7eea00 Sep 17 15:56:53.351471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.351453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:56:53.351535 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.351515 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:53.351605 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.351586 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 15:56:53.351702 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.351638 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls podName:9ac10347-81ed-4b23-8eac-ddc2a4c1d171 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:54.351625316 +0000 UTC m=+71.454870675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-5fpms" (UID: "9ac10347-81ed-4b23-8eac-ddc2a4c1d171") : secret "cluster-monitoring-operator-tls" not found Sep 17 15:56:53.351702 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.351657 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 15:56:53.351702 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.351672 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f94b94954-h5n4x: secret "image-registry-tls" not found Sep 17 15:56:53.351860 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.351712 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls podName:0a11d802-951f-4a33-bb26-7b2d2b641ca6 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:54.351701071 +0000 UTC m=+71.454946431 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls") pod "image-registry-6f94b94954-h5n4x" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6") : secret "image-registry-tls" not found Sep 17 15:56:53.361404 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.361382 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh"] Sep 17 15:56:53.364860 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:56:53.364834 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4861c6ad_a4a3_4947_a1f1_b21601e7a455.slice/crio-6cd0be32e19af368d9e33a6114a363d499b0274ed62750c7be1561f597fda320 WatchSource:0}: Error finding container 6cd0be32e19af368d9e33a6114a363d499b0274ed62750c7be1561f597fda320: Status 404 returned error can't find the container with id 6cd0be32e19af368d9e33a6114a363d499b0274ed62750c7be1561f597fda320 Sep 17 15:56:53.390536 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.390513 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-pnkdl"] Sep 17 15:56:53.393607 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:56:53.393582 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod790edfda_48c6_4852_9b6f_e238976373be.slice/crio-5d1dedd9ae99b0cbe359e660814f7dd78f8cf4a8cf708e4ecd7bf4f44391fd0d WatchSource:0}: Error finding container 5d1dedd9ae99b0cbe359e660814f7dd78f8cf4a8cf708e4ecd7bf4f44391fd0d: Status 404 returned error can't find the container with id 5d1dedd9ae99b0cbe359e660814f7dd78f8cf4a8cf708e4ecd7bf4f44391fd0d Sep 17 15:56:53.412287 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.412266 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-fj294"] Sep 17 15:56:53.414910 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.414893 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fj294" Sep 17 15:56:53.417500 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.417480 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wzwjh\"" Sep 17 15:56:53.424465 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.424448 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r"] Sep 17 15:56:53.426536 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:56:53.426512 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf62b1102_06d5_4996_bdcc_f75199fe0464.slice/crio-379a62c410830927947b37e0099f0941eb6c53a59fe646314b88f84523444d28 WatchSource:0}: Error finding container 379a62c410830927947b37e0099f0941eb6c53a59fe646314b88f84523444d28: Status 404 returned error can't find the container with id 379a62c410830927947b37e0099f0941eb6c53a59fe646314b88f84523444d28 Sep 17 15:56:53.437036 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.435188 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-88zv4"] Sep 17 15:56:53.452293 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.452267 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-9f49r\" (UID: \"7ad114b5-1283-442f-b7b2-fde01ae0f839\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:56:53.452418 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.452304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c7b711c-7da3-4f63-ade9-9282296f6086-tmp-dir\") pod \"node-resolver-fj294\" (UID: \"3c7b711c-7da3-4f63-ade9-9282296f6086\") " pod="openshift-dns/node-resolver-fj294" Sep 17 15:56:53.452418 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.452327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:53.452418 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.452354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:53.452592 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.452424 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 15:56:53.452592 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.452486 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls podName:7ad114b5-1283-442f-b7b2-fde01ae0f839 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:54.452467014 +0000 UTC m=+71.555712380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-9f49r" (UID: "7ad114b5-1283-442f-b7b2-fde01ae0f839") : secret "samples-operator-tls" not found Sep 17 15:56:53.452592 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.452493 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 15:56:53.452753 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.452595 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls podName:d24009de-229f-4bb3-a49e-5499cfc0b519 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:54.452580946 +0000 UTC m=+71.555826313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls") pod "dns-default-2bdcw" (UID: "d24009de-229f-4bb3-a49e-5499cfc0b519") : secret "dns-default-metrics-tls" not found Sep 17 15:56:53.452753 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.452622 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle podName:99684b4e-9484-4867-a730-ea1662218945 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:54.452611205 +0000 UTC m=+71.555856565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle") pod "router-default-86dfcf9bc6-7j9bp" (UID: "99684b4e-9484-4867-a730-ea1662218945") : configmap references non-existent config key: service-ca.crt Sep 17 15:56:53.452753 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.452706 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:53.452907 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.452769 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ghjkd\" (UID: \"51b96393-ed39-45b3-baca-b0acb42c254b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:56:53.452907 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.452815 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6xm6\" (UniqueName: \"kubernetes.io/projected/3c7b711c-7da3-4f63-ade9-9282296f6086-kube-api-access-h6xm6\") pod \"node-resolver-fj294\" (UID: \"3c7b711c-7da3-4f63-ade9-9282296f6086\") " pod="openshift-dns/node-resolver-fj294" Sep 17 15:56:53.452907 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.452845 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3c7b711c-7da3-4f63-ade9-9282296f6086-hosts-file\") pod \"node-resolver-fj294\" (UID: \"3c7b711c-7da3-4f63-ade9-9282296f6086\") " pod="openshift-dns/node-resolver-fj294" Sep 17 15:56:53.452907 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.452848 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 15:56:53.452907 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.452898 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 15:56:53.453081 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.452937 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert podName:51b96393-ed39-45b3-baca-b0acb42c254b nodeName:}" failed. No retries permitted until 2026-09-17 15:56:54.452923628 +0000 UTC m=+71.556168993 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-ghjkd" (UID: "51b96393-ed39-45b3-baca-b0acb42c254b") : secret "networking-console-plugin-cert" not found Sep 17 15:56:53.453081 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:53.452965 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs podName:99684b4e-9484-4867-a730-ea1662218945 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:54.452952964 +0000 UTC m=+71.556198345 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs") pod "router-default-86dfcf9bc6-7j9bp" (UID: "99684b4e-9484-4867-a730-ea1662218945") : secret "router-metrics-certs-default" not found Sep 17 15:56:53.506673 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:56:53.506645 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod49f537d0_1508_448d_af52_bc6b8fe22ece.slice/crio-9ac087101a316b9f935f64fb6b2f0b640366e25f6044d39df1e5c7a07b0e898b WatchSource:0}: Error finding container 9ac087101a316b9f935f64fb6b2f0b640366e25f6044d39df1e5c7a07b0e898b: Status 404 returned error can't find the container with id 9ac087101a316b9f935f64fb6b2f0b640366e25f6044d39df1e5c7a07b0e898b Sep 17 15:56:53.510363 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:56:53.510338 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15601992_7dfc_4244_848a_eccecf6bc1c5.slice/crio-28df5dc4f83f4d92acd3d4b98dbf363ce61718ad86ebb7fd37212eff63dca07b WatchSource:0}: Error finding container 28df5dc4f83f4d92acd3d4b98dbf363ce61718ad86ebb7fd37212eff63dca07b: Status 404 returned error can't find the container with id 28df5dc4f83f4d92acd3d4b98dbf363ce61718ad86ebb7fd37212eff63dca07b Sep 17 15:56:53.513078 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.513055 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh"] Sep 17 15:56:53.513910 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.513891 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5lzg5"] Sep 17 15:56:53.540139 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.540108 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-fd4bb46c-qp4q9"] Sep 17 15:56:53.540932 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:56:53.540910 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a3b84c4_ac8d_4e8a_a606_aba79b7fe148.slice/crio-f148fed99de57905ec5abeb50f87008497527a388ad8f9a0af932e596b54940a WatchSource:0}: Error finding container f148fed99de57905ec5abeb50f87008497527a388ad8f9a0af932e596b54940a: Status 404 returned error can't find the container with id f148fed99de57905ec5abeb50f87008497527a388ad8f9a0af932e596b54940a Sep 17 15:56:53.553129 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.553109 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:56:53.553320 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.553304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6xm6\" (UniqueName: \"kubernetes.io/projected/3c7b711c-7da3-4f63-ade9-9282296f6086-kube-api-access-h6xm6\") pod \"node-resolver-fj294\" (UID: \"3c7b711c-7da3-4f63-ade9-9282296f6086\") " pod="openshift-dns/node-resolver-fj294" Sep 17 15:56:53.553372 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.553331 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3c7b711c-7da3-4f63-ade9-9282296f6086-hosts-file\") pod \"node-resolver-fj294\" (UID: \"3c7b711c-7da3-4f63-ade9-9282296f6086\") " pod="openshift-dns/node-resolver-fj294" Sep 17 15:56:53.553372 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.553362 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c7b711c-7da3-4f63-ade9-9282296f6086-tmp-dir\") pod \"node-resolver-fj294\" (UID: \"3c7b711c-7da3-4f63-ade9-9282296f6086\") " pod="openshift-dns/node-resolver-fj294" Sep 17 15:56:53.553451 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.553434 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3c7b711c-7da3-4f63-ade9-9282296f6086-hosts-file\") pod \"node-resolver-fj294\" (UID: \"3c7b711c-7da3-4f63-ade9-9282296f6086\") " pod="openshift-dns/node-resolver-fj294" Sep 17 15:56:53.553596 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.553583 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c7b711c-7da3-4f63-ade9-9282296f6086-tmp-dir\") pod \"node-resolver-fj294\" (UID: \"3c7b711c-7da3-4f63-ade9-9282296f6086\") " pod="openshift-dns/node-resolver-fj294" Sep 17 15:56:53.561002 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.560984 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz"] Sep 17 15:56:53.563656 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:56:53.563633 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52fe5bcc_c834_460f_bf01_cc292c18c146.slice/crio-656b076a9ac043b059ea64e05b4974ac9f09c7e38b00c7f8e57260903259e027 WatchSource:0}: Error finding container 656b076a9ac043b059ea64e05b4974ac9f09c7e38b00c7f8e57260903259e027: Status 404 returned error can't find the container with id 656b076a9ac043b059ea64e05b4974ac9f09c7e38b00c7f8e57260903259e027 Sep 17 15:56:53.573745 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.573728 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6xm6\" (UniqueName: \"kubernetes.io/projected/3c7b711c-7da3-4f63-ade9-9282296f6086-kube-api-access-h6xm6\") pod \"node-resolver-fj294\" (UID: \"3c7b711c-7da3-4f63-ade9-9282296f6086\") " pod="openshift-dns/node-resolver-fj294" Sep 17 15:56:53.680152 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.680123 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:56:53.724325 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.724293 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fj294" Sep 17 15:56:53.734604 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:56:53.734580 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c7b711c_7da3_4f63_ade9_9282296f6086.slice/crio-801e1d9d2d384f7835e12f91e8abc267b85db263e70b79267c9af6697b5698e8 WatchSource:0}: Error finding container 801e1d9d2d384f7835e12f91e8abc267b85db263e70b79267c9af6697b5698e8: Status 404 returned error can't find the container with id 801e1d9d2d384f7835e12f91e8abc267b85db263e70b79267c9af6697b5698e8 Sep 17 15:56:53.807643 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.807613 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" event={"ID":"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4","Type":"ContainerStarted","Data":"9e5c3c2406eef98b2d625dd9b7c54aaa7d1a5af7c8b6242f5b76dddb0f7eea00"} Sep 17 15:56:53.808811 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.808785 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" event={"ID":"4861c6ad-a4a3-4947-a1f1-b21601e7a455","Type":"ContainerStarted","Data":"6cd0be32e19af368d9e33a6114a363d499b0274ed62750c7be1561f597fda320"} Sep 17 15:56:53.809965 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.809944 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" event={"ID":"52fe5bcc-c834-460f-bf01-cc292c18c146","Type":"ContainerStarted","Data":"656b076a9ac043b059ea64e05b4974ac9f09c7e38b00c7f8e57260903259e027"} Sep 17 15:56:53.811051 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.811033 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-88zv4" event={"ID":"5f94a7e8-2cf1-4051-a1d5-c7d29241fab3","Type":"ContainerStarted","Data":"e92da9c9050f72216c6e240c14e8e9214d1f11f66c1a7de197c7334d4ba2948d"} Sep 17 15:56:53.812119 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.812092 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-pnkdl" event={"ID":"790edfda-48c6-4852-9b6f-e238976373be","Type":"ContainerStarted","Data":"5d1dedd9ae99b0cbe359e660814f7dd78f8cf4a8cf708e4ecd7bf4f44391fd0d"} Sep 17 15:56:53.813286 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.813242 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fj294" event={"ID":"3c7b711c-7da3-4f63-ade9-9282296f6086","Type":"ContainerStarted","Data":"801e1d9d2d384f7835e12f91e8abc267b85db263e70b79267c9af6697b5698e8"} Sep 17 15:56:53.814650 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.814522 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" event={"ID":"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148","Type":"ContainerStarted","Data":"f148fed99de57905ec5abeb50f87008497527a388ad8f9a0af932e596b54940a"} Sep 17 15:56:53.815892 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.815868 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5lzg5" event={"ID":"15601992-7dfc-4244-848a-eccecf6bc1c5","Type":"ContainerStarted","Data":"28df5dc4f83f4d92acd3d4b98dbf363ce61718ad86ebb7fd37212eff63dca07b"} Sep 17 15:56:53.817384 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.817324 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" event={"ID":"49f537d0-1508-448d-af52-bc6b8fe22ece","Type":"ContainerStarted","Data":"9ac087101a316b9f935f64fb6b2f0b640366e25f6044d39df1e5c7a07b0e898b"} Sep 17 15:56:53.818620 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.818599 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r" event={"ID":"f62b1102-06d5-4996-bdcc-f75199fe0464","Type":"ContainerStarted","Data":"379a62c410830927947b37e0099f0941eb6c53a59fe646314b88f84523444d28"} Sep 17 15:56:53.819930 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:53.819891 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" event={"ID":"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b","Type":"ContainerStarted","Data":"d312f886582552e9852f24a4cab757cdfb5980123edc7b2ac145e6da5d76a99f"} Sep 17 15:56:54.364452 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.364414 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:56:54.364627 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.364511 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:54.364688 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.364637 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 15:56:54.364688 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.364653 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f94b94954-h5n4x: secret "image-registry-tls" not found Sep 17 15:56:54.364948 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.364713 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls podName:0a11d802-951f-4a33-bb26-7b2d2b641ca6 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:56.364694065 +0000 UTC m=+73.467939425 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls") pod "image-registry-6f94b94954-h5n4x" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6") : secret "image-registry-tls" not found Sep 17 15:56:54.365130 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.365100 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 15:56:54.365298 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.365161 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls podName:9ac10347-81ed-4b23-8eac-ddc2a4c1d171 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:56.365146143 +0000 UTC m=+73.468391503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-5fpms" (UID: "9ac10347-81ed-4b23-8eac-ddc2a4c1d171") : secret "cluster-monitoring-operator-tls" not found Sep 17 15:56:54.465089 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.465048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-9f49r\" (UID: \"7ad114b5-1283-442f-b7b2-fde01ae0f839\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:56:54.465331 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.465119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:54.465331 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.465170 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:54.465331 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.465258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:54.465331 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.465313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ghjkd\" (UID: \"51b96393-ed39-45b3-baca-b0acb42c254b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:56:54.465551 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.465449 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 15:56:54.465551 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.465509 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert podName:51b96393-ed39-45b3-baca-b0acb42c254b nodeName:}" failed. No retries permitted until 2026-09-17 15:56:56.465490392 +0000 UTC m=+73.568735775 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-ghjkd" (UID: "51b96393-ed39-45b3-baca-b0acb42c254b") : secret "networking-console-plugin-cert" not found Sep 17 15:56:54.465920 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.465903 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 15:56:54.465985 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.465962 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls podName:7ad114b5-1283-442f-b7b2-fde01ae0f839 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:56.465940801 +0000 UTC m=+73.569186167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-9f49r" (UID: "7ad114b5-1283-442f-b7b2-fde01ae0f839") : secret "samples-operator-tls" not found Sep 17 15:56:54.466044 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.466029 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle podName:99684b4e-9484-4867-a730-ea1662218945 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:56.466018743 +0000 UTC m=+73.569264108 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle") pod "router-default-86dfcf9bc6-7j9bp" (UID: "99684b4e-9484-4867-a730-ea1662218945") : configmap references non-existent config key: service-ca.crt Sep 17 15:56:54.466104 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.466082 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 15:56:54.466160 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.466109 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls podName:d24009de-229f-4bb3-a49e-5499cfc0b519 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:56.466099516 +0000 UTC m=+73.569344881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls") pod "dns-default-2bdcw" (UID: "d24009de-229f-4bb3-a49e-5499cfc0b519") : secret "dns-default-metrics-tls" not found Sep 17 15:56:54.466241 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.466159 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 15:56:54.466241 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:54.466188 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs podName:99684b4e-9484-4867-a730-ea1662218945 nodeName:}" failed. No retries permitted until 2026-09-17 15:56:56.466178673 +0000 UTC m=+73.569424039 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs") pod "router-default-86dfcf9bc6-7j9bp" (UID: "99684b4e-9484-4867-a730-ea1662218945") : secret "router-metrics-certs-default" not found Sep 17 15:56:54.488442 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.488416 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:56:54.488854 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.488837 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:56:54.489257 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.489242 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:56:54.492154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.492129 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 15:56:54.493409 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.493362 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 15:56:54.493691 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.493653 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fvxq9\"" Sep 17 15:56:54.493957 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.493938 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v87nd\"" Sep 17 15:56:54.779775 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.779697 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:56:54.849997 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.849929 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fj294" event={"ID":"3c7b711c-7da3-4f63-ade9-9282296f6086","Type":"ContainerStarted","Data":"1552d843f1a2830281fce8be0d230befe42d00cc2184d90a1bb43f2e7aa7fa77"} Sep 17 15:56:54.877611 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:54.877577 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:56:55.396733 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:55.396604 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:56:55.501976 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:55.501930 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:56:56.391581 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:56.390697 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:56:56.391581 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:56.390848 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:56:56.391581 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.391028 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 15:56:56.391581 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.391089 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls podName:9ac10347-81ed-4b23-8eac-ddc2a4c1d171 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:00.391071068 +0000 UTC m=+77.494316430 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-5fpms" (UID: "9ac10347-81ed-4b23-8eac-ddc2a4c1d171") : secret "cluster-monitoring-operator-tls" not found Sep 17 15:56:56.391581 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.391488 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 15:56:56.391581 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.391505 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f94b94954-h5n4x: secret "image-registry-tls" not found Sep 17 15:56:56.391581 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.391549 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls podName:0a11d802-951f-4a33-bb26-7b2d2b641ca6 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:00.391534168 +0000 UTC m=+77.494779537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls") pod "image-registry-6f94b94954-h5n4x" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6") : secret "image-registry-tls" not found Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:56.492705 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:56.492782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ghjkd\" (UID: \"51b96393-ed39-45b3-baca-b0acb42c254b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:56.492837 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-9f49r\" (UID: \"7ad114b5-1283-442f-b7b2-fde01ae0f839\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:56.492886 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:56.492933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.493091 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.493095 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.493151 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls podName:7ad114b5-1283-442f-b7b2-fde01ae0f839 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:00.493131353 +0000 UTC m=+77.596376729 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-9f49r" (UID: "7ad114b5-1283-442f-b7b2-fde01ae0f839") : secret "samples-operator-tls" not found Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.493169 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert podName:51b96393-ed39-45b3-baca-b0acb42c254b nodeName:}" failed. No retries permitted until 2026-09-17 15:57:00.493159429 +0000 UTC m=+77.596404794 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-ghjkd" (UID: "51b96393-ed39-45b3-baca-b0acb42c254b") : secret "networking-console-plugin-cert" not found Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.493197 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.493254 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls podName:d24009de-229f-4bb3-a49e-5499cfc0b519 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:00.493236017 +0000 UTC m=+77.596481392 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls") pod "dns-default-2bdcw" (UID: "d24009de-229f-4bb3-a49e-5499cfc0b519") : secret "dns-default-metrics-tls" not found Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.493274 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle podName:99684b4e-9484-4867-a730-ea1662218945 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:00.493264218 +0000 UTC m=+77.596509583 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle") pod "router-default-86dfcf9bc6-7j9bp" (UID: "99684b4e-9484-4867-a730-ea1662218945") : configmap references non-existent config key: service-ca.crt Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.493304 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 15:56:56.493361 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:56:56.493335 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs podName:99684b4e-9484-4867-a730-ea1662218945 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:00.493325403 +0000 UTC m=+77.596570766 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs") pod "router-default-86dfcf9bc6-7j9bp" (UID: "99684b4e-9484-4867-a730-ea1662218945") : secret "router-metrics-certs-default" not found Sep 17 15:56:56.881378 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:56.881343 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5lzg5" event={"ID":"15601992-7dfc-4244-848a-eccecf6bc1c5","Type":"ContainerStarted","Data":"1e831bb41820cbc1949da8d70e3c13d9cd131837ec2262f7afee05fcce42bb9a"} Sep 17 15:56:56.903822 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:56.903768 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fj294" podStartSLOduration=3.903747093 podStartE2EDuration="3.903747093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:56:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:54.874912521 +0000 UTC m=+71.978157909" watchObservedRunningTime="2026-09-17 15:56:56.903747093 +0000 UTC m=+74.006992477" Sep 17 15:56:57.049174 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:56:57.049139 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:00.113172 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:00.113122 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:00.186647 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:00.186607 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:00.432963 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:00.432872 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:57:00.433145 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.433081 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 15:57:00.433145 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:00.433098 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:57:00.433301 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.433162 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls podName:9ac10347-81ed-4b23-8eac-ddc2a4c1d171 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:08.433139297 +0000 UTC m=+85.536384657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-5fpms" (UID: "9ac10347-81ed-4b23-8eac-ddc2a4c1d171") : secret "cluster-monitoring-operator-tls" not found Sep 17 15:57:00.433301 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.433237 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 15:57:00.433301 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.433254 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f94b94954-h5n4x: secret "image-registry-tls" not found Sep 17 15:57:00.433422 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.433316 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls podName:0a11d802-951f-4a33-bb26-7b2d2b641ca6 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:08.433296834 +0000 UTC m=+85.536542197 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls") pod "image-registry-6f94b94954-h5n4x" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6") : secret "image-registry-tls" not found Sep 17 15:57:00.534689 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:00.534655 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:00.534857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:00.534729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ghjkd\" (UID: \"51b96393-ed39-45b3-baca-b0acb42c254b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:57:00.534857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:00.534786 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-9f49r\" (UID: \"7ad114b5-1283-442f-b7b2-fde01ae0f839\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:57:00.534857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:00.534830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:00.534980 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:00.534859 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:57:00.534980 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.534864 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 15:57:00.534980 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.534910 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 15:57:00.534980 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.534943 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs podName:99684b4e-9484-4867-a730-ea1662218945 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:08.534921959 +0000 UTC m=+85.638167334 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs") pod "router-default-86dfcf9bc6-7j9bp" (UID: "99684b4e-9484-4867-a730-ea1662218945") : secret "router-metrics-certs-default" not found Sep 17 15:57:00.534980 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.534969 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 15:57:00.535180 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.534991 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 15:57:00.535180 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.534973 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert podName:51b96393-ed39-45b3-baca-b0acb42c254b nodeName:}" failed. No retries permitted until 2026-09-17 15:57:08.53495532 +0000 UTC m=+85.638200685 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-ghjkd" (UID: "51b96393-ed39-45b3-baca-b0acb42c254b") : secret "networking-console-plugin-cert" not found Sep 17 15:57:00.535180 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.535020 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle podName:99684b4e-9484-4867-a730-ea1662218945 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:08.535013734 +0000 UTC m=+85.638259094 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle") pod "router-default-86dfcf9bc6-7j9bp" (UID: "99684b4e-9484-4867-a730-ea1662218945") : configmap references non-existent config key: service-ca.crt Sep 17 15:57:00.535180 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.535035 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls podName:7ad114b5-1283-442f-b7b2-fde01ae0f839 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:08.535029003 +0000 UTC m=+85.638274362 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-9f49r" (UID: "7ad114b5-1283-442f-b7b2-fde01ae0f839") : secret "samples-operator-tls" not found Sep 17 15:57:00.535180 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:00.535045 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls podName:d24009de-229f-4bb3-a49e-5499cfc0b519 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:08.53503975 +0000 UTC m=+85.638285110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls") pod "dns-default-2bdcw" (UID: "d24009de-229f-4bb3-a49e-5499cfc0b519") : secret "dns-default-metrics-tls" not found Sep 17 15:57:05.911472 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.911431 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-88zv4" event={"ID":"5f94a7e8-2cf1-4051-a1d5-c7d29241fab3","Type":"ContainerStarted","Data":"00994da7f974994c8f1a24783683a65892b2431ea0abe2db2b24cdbce181c00e"} Sep 17 15:57:05.913490 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.913439 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-pnkdl" event={"ID":"790edfda-48c6-4852-9b6f-e238976373be","Type":"ContainerStarted","Data":"cac56b4e99b7770491f7db9be6afc398cd867bb8d1771e2d2e0dcce9dd2bb050"} Sep 17 15:57:05.915355 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.915324 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/0.log" Sep 17 15:57:05.915479 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.915372 2580 generic.go:356] "Generic (PLEG): container finished" podID="0a3b84c4-ac8d-4e8a-a606-aba79b7fe148" containerID="0c4c604dd315d8029bcb83fc89badf676618561ac06c5b08c5f2246f074262e3" exitCode=255 Sep 17 15:57:05.915479 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.915454 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" event={"ID":"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148","Type":"ContainerDied","Data":"0c4c604dd315d8029bcb83fc89badf676618561ac06c5b08c5f2246f074262e3"} Sep 17 15:57:05.915935 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.915917 2580 scope.go:117] "RemoveContainer" containerID="0c4c604dd315d8029bcb83fc89badf676618561ac06c5b08c5f2246f074262e3" Sep 17 15:57:05.918539 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.917979 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" event={"ID":"49f537d0-1508-448d-af52-bc6b8fe22ece","Type":"ContainerStarted","Data":"9723bbe82e151537b674d76165cc52fb26169515ec6cd1283c092f427c997aa4"} Sep 17 15:57:05.918539 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.918017 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" event={"ID":"49f537d0-1508-448d-af52-bc6b8fe22ece","Type":"ContainerStarted","Data":"3570b737fe04f6a887d565864a1fafb500349cb6a2b9cb9afb27d1e3e1934d5c"} Sep 17 15:57:05.918539 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.918035 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" event={"ID":"49f537d0-1508-448d-af52-bc6b8fe22ece","Type":"ContainerStarted","Data":"69e42e2661034a8d2afc41edb187b90b5c92b047350b135ee9892c8581b70bcb"} Sep 17 15:57:05.920451 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.920422 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r" event={"ID":"f62b1102-06d5-4996-bdcc-f75199fe0464","Type":"ContainerStarted","Data":"3cf36c219b104de12315d539295550311d0805b05c0410626606352e0de08b9e"} Sep 17 15:57:05.922488 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.922453 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" event={"ID":"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b","Type":"ContainerStarted","Data":"a7d1586a1402111884dbba0ae43c017cbf3bfec641f42f0baaa5c04a9fe41078"} Sep 17 15:57:05.924299 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.924280 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" event={"ID":"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4","Type":"ContainerStarted","Data":"62708dfa5ca10dbceb2d4774e31d617d57533384e3c395a9c70d92b74a72acdd"} Sep 17 15:57:05.926001 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.925980 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" event={"ID":"4861c6ad-a4a3-4947-a1f1-b21601e7a455","Type":"ContainerStarted","Data":"0250c48096b52de0dd35eaad819cedde996b39342027dd61106f235db274a08b"} Sep 17 15:57:05.927629 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.927606 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" event={"ID":"52fe5bcc-c834-460f-bf01-cc292c18c146","Type":"ContainerStarted","Data":"090dd852ea7e5841b520b1163ed432bf6854bd7bf359954592d2406374bb3ceb"} Sep 17 15:57:05.927912 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.927883 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:57:05.929683 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.929666 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" Sep 17 15:57:05.934524 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.934468 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-5lzg5" podStartSLOduration=93.097893326 podStartE2EDuration="1m35.934452198s" totalImagesPullingTime="2.836558872s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:56:56.905087604 +0000 UTC m=+74.008332988" watchObservedRunningTime="2026-09-17 15:57:05.934452198 +0000 UTC m=+83.037697583" Sep 17 15:57:05.935996 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.934730 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-88zv4" podStartSLOduration=2.050569452 podStartE2EDuration="13.93471674s" totalImagesPullingTime="11.884147288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:05.934290895 +0000 UTC m=+83.037536275" watchObservedRunningTime="2026-09-17 15:57:05.93471674 +0000 UTC m=+83.037962123" Sep 17 15:57:05.957341 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:05.957252 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" podStartSLOduration=62.122800569 podStartE2EDuration="1m13.957208273s" totalImagesPullingTime="11.834407704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:05.954751742 +0000 UTC m=+83.057997123" watchObservedRunningTime="2026-09-17 15:57:05.957208273 +0000 UTC m=+83.060453656" Sep 17 15:57:06.002192 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.000869 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-pnkdl" podStartSLOduration=84.071672089 podStartE2EDuration="1m36.000847445s" totalImagesPullingTime="11.929175356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:05.979000505 +0000 UTC m=+83.082245887" watchObservedRunningTime="2026-09-17 15:57:06.000847445 +0000 UTC m=+83.104092828" Sep 17 15:57:06.002192 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.000969 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-858cb56476-tdh4r" podStartSLOduration=62.104408977 podStartE2EDuration="1m14.000960316s" totalImagesPullingTime="11.896551339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:05.999812331 +0000 UTC m=+83.103057715" watchObservedRunningTime="2026-09-17 15:57:06.000960316 +0000 UTC m=+83.104205700" Sep 17 15:57:06.021007 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.020255 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" podStartSLOduration=84.045455025 podStartE2EDuration="1m36.020233521s" totalImagesPullingTime="11.974778496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:06.01949073 +0000 UTC m=+83.122736112" watchObservedRunningTime="2026-09-17 15:57:06.020233521 +0000 UTC m=+83.123478898" Sep 17 15:57:06.062925 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.062856 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5c44cc5f4-s5fxz" podStartSLOduration=62.303865415 podStartE2EDuration="1m14.062837134s" totalImagesPullingTime="11.758971719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:06.061595214 +0000 UTC m=+83.164840592" watchObservedRunningTime="2026-09-17 15:57:06.062837134 +0000 UTC m=+83.166082518" Sep 17 15:57:06.084562 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.084510 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" podStartSLOduration=84.110562034 podStartE2EDuration="1m36.084481066s" totalImagesPullingTime="11.973919032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:06.083746794 +0000 UTC m=+83.186992177" watchObservedRunningTime="2026-09-17 15:57:06.084481066 +0000 UTC m=+83.187726450" Sep 17 15:57:06.118874 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.118819 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" podStartSLOduration=84.16264759 podStartE2EDuration="1m36.11880344s" totalImagesPullingTime="11.95615585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:06.11875285 +0000 UTC m=+83.221998234" watchObservedRunningTime="2026-09-17 15:57:06.11880344 +0000 UTC m=+83.222048821" Sep 17 15:57:06.732482 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.732439 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:06.866379 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.866343 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:06.933854 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.933827 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 15:57:06.934335 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.934320 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/0.log" Sep 17 15:57:06.934390 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.934357 2580 generic.go:356] "Generic (PLEG): container finished" podID="0a3b84c4-ac8d-4e8a-a606-aba79b7fe148" containerID="4d3c8231d51ae89526e729b570a19bbf305db9112f03629bb87bede7dfb36658" exitCode=255 Sep 17 15:57:06.935158 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.935141 2580 scope.go:117] "RemoveContainer" containerID="4d3c8231d51ae89526e729b570a19bbf305db9112f03629bb87bede7dfb36658" Sep 17 15:57:06.935345 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:06.935318 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-qp4q9_openshift-console-operator(0a3b84c4-ac8d-4e8a-a606-aba79b7fe148)\"" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" podUID="0a3b84c4-ac8d-4e8a-a606-aba79b7fe148" Sep 17 15:57:06.935802 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.935780 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" event={"ID":"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148","Type":"ContainerDied","Data":"4d3c8231d51ae89526e729b570a19bbf305db9112f03629bb87bede7dfb36658"} Sep 17 15:57:06.935985 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:06.935973 2580 scope.go:117] "RemoveContainer" containerID="0c4c604dd315d8029bcb83fc89badf676618561ac06c5b08c5f2246f074262e3" Sep 17 15:57:07.035684 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.035599 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:07.144154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.144102 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:07.256470 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.256438 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:07.452966 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.452935 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:07.637253 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.637203 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:07.659297 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.659270 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6fskk"] Sep 17 15:57:07.662600 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.662581 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.672543 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.672523 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5b2qd\"" Sep 17 15:57:07.672721 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.672707 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 17 15:57:07.673300 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.673288 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 17 15:57:07.693867 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.693847 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6fskk"] Sep 17 15:57:07.808382 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.808350 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:07.810497 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.810478 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b832eb9-54df-42b4-8e08-a3f28c1f598e-data-volume\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.810564 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.810518 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.810626 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.810606 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxclt\" (UniqueName: \"kubernetes.io/projected/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-api-access-nxclt\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.810676 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.810663 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1b832eb9-54df-42b4-8e08-a3f28c1f598e-crio-socket\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.810707 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.810695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.911617 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.911580 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1b832eb9-54df-42b4-8e08-a3f28c1f598e-crio-socket\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.911617 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.911621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.911785 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.911697 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b832eb9-54df-42b4-8e08-a3f28c1f598e-data-volume\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.911785 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.911725 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.911785 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.911721 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1b832eb9-54df-42b4-8e08-a3f28c1f598e-crio-socket\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.911943 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:07.911782 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 15:57:07.911943 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.911822 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxclt\" (UniqueName: \"kubernetes.io/projected/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-api-access-nxclt\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.911943 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:07.911825 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 15:57:07.911943 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:07.911877 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm podName:1b832eb9-54df-42b4-8e08-a3f28c1f598e nodeName:}" failed. No retries permitted until 2026-09-17 15:57:08.41185774 +0000 UTC m=+85.515103145 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6fskk" (UID: "1b832eb9-54df-42b4-8e08-a3f28c1f598e") : configmap "kube-rbac-proxy" not found Sep 17 15:57:07.911943 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:07.911931 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls podName:1b832eb9-54df-42b4-8e08-a3f28c1f598e nodeName:}" failed. No retries permitted until 2026-09-17 15:57:08.411923691 +0000 UTC m=+85.515169052 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6fskk" (UID: "1b832eb9-54df-42b4-8e08-a3f28c1f598e") : secret "insights-runtime-extractor-tls" not found Sep 17 15:57:07.912089 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.912050 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1b832eb9-54df-42b4-8e08-a3f28c1f598e-data-volume\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.929671 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.929647 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxclt\" (UniqueName: \"kubernetes.io/projected/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-api-access-nxclt\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:07.939301 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.939278 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 15:57:07.939640 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:07.939622 2580 scope.go:117] "RemoveContainer" containerID="4d3c8231d51ae89526e729b570a19bbf305db9112f03629bb87bede7dfb36658" Sep 17 15:57:07.939801 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:07.939780 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-qp4q9_openshift-console-operator(0a3b84c4-ac8d-4e8a-a606-aba79b7fe148)\"" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" podUID="0a3b84c4-ac8d-4e8a-a606-aba79b7fe148" Sep 17 15:57:08.417741 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:08.417706 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:08.417903 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.417829 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 15:57:08.417903 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.417895 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm podName:1b832eb9-54df-42b4-8e08-a3f28c1f598e nodeName:}" failed. No retries permitted until 2026-09-17 15:57:09.417881673 +0000 UTC m=+86.521127037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6fskk" (UID: "1b832eb9-54df-42b4-8e08-a3f28c1f598e") : configmap "kube-rbac-proxy" not found Sep 17 15:57:08.417980 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:08.417926 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:08.418029 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.418018 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 15:57:08.418063 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.418050 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls podName:1b832eb9-54df-42b4-8e08-a3f28c1f598e nodeName:}" failed. No retries permitted until 2026-09-17 15:57:09.418042606 +0000 UTC m=+86.521287966 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6fskk" (UID: "1b832eb9-54df-42b4-8e08-a3f28c1f598e") : secret "insights-runtime-extractor-tls" not found Sep 17 15:57:08.463116 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:08.463085 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:08.518788 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:08.518755 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:57:08.519003 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:08.518848 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:57:08.519003 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.518898 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 17 15:57:08.519003 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.518964 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls podName:9ac10347-81ed-4b23-8eac-ddc2a4c1d171 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:24.518949736 +0000 UTC m=+101.622195096 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-847b8bd8d7-5fpms" (UID: "9ac10347-81ed-4b23-8eac-ddc2a4c1d171") : secret "cluster-monitoring-operator-tls" not found Sep 17 15:57:08.519003 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.518965 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 17 15:57:08.519003 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.518978 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6f94b94954-h5n4x: secret "image-registry-tls" not found Sep 17 15:57:08.519158 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.519028 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls podName:0a11d802-951f-4a33-bb26-7b2d2b641ca6 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:24.519013598 +0000 UTC m=+101.622258959 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls") pod "image-registry-6f94b94954-h5n4x" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6") : secret "image-registry-tls" not found Sep 17 15:57:08.572913 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:08.572888 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:08.619682 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:08.619647 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-9f49r\" (UID: \"7ad114b5-1283-442f-b7b2-fde01ae0f839\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:57:08.619813 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:08.619695 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:08.619813 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.619785 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle podName:99684b4e-9484-4867-a730-ea1662218945 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:24.619772998 +0000 UTC m=+101.723018358 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle") pod "router-default-86dfcf9bc6-7j9bp" (UID: "99684b4e-9484-4867-a730-ea1662218945") : configmap references non-existent config key: service-ca.crt Sep 17 15:57:08.619813 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.619800 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 17 15:57:08.619921 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.619852 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls podName:7ad114b5-1283-442f-b7b2-fde01ae0f839 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:24.619838586 +0000 UTC m=+101.723083946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls") pod "cluster-samples-operator-685d7d68b5-9f49r" (UID: "7ad114b5-1283-442f-b7b2-fde01ae0f839") : secret "samples-operator-tls" not found Sep 17 15:57:08.619921 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:08.619808 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:57:08.619987 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.619863 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 17 15:57:08.619987 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:08.619936 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:08.619987 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.619964 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls podName:d24009de-229f-4bb3-a49e-5499cfc0b519 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:24.619951783 +0000 UTC m=+101.723197147 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls") pod "dns-default-2bdcw" (UID: "d24009de-229f-4bb3-a49e-5499cfc0b519") : secret "dns-default-metrics-tls" not found Sep 17 15:57:08.620075 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:08.619998 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ghjkd\" (UID: \"51b96393-ed39-45b3-baca-b0acb42c254b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:57:08.620075 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.620010 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 17 15:57:08.620075 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.620059 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs podName:99684b4e-9484-4867-a730-ea1662218945 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:24.620045753 +0000 UTC m=+101.723291128 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs") pod "router-default-86dfcf9bc6-7j9bp" (UID: "99684b4e-9484-4867-a730-ea1662218945") : secret "router-metrics-certs-default" not found Sep 17 15:57:08.620160 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.620085 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 17 15:57:08.620160 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:08.620122 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert podName:51b96393-ed39-45b3-baca-b0acb42c254b nodeName:}" failed. No retries permitted until 2026-09-17 15:57:24.620110902 +0000 UTC m=+101.723356264 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-ghjkd" (UID: "51b96393-ed39-45b3-baca-b0acb42c254b") : secret "networking-console-plugin-cert" not found Sep 17 15:57:09.073004 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:09.072970 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:09.170775 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:09.170737 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:09.428277 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:09.428189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:09.428423 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:09.428318 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 15:57:09.428423 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:09.428366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:09.428423 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:09.428403 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm podName:1b832eb9-54df-42b4-8e08-a3f28c1f598e nodeName:}" failed. No retries permitted until 2026-09-17 15:57:11.428384754 +0000 UTC m=+88.531630137 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6fskk" (UID: "1b832eb9-54df-42b4-8e08-a3f28c1f598e") : configmap "kube-rbac-proxy" not found Sep 17 15:57:09.428560 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:09.428469 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 15:57:09.428560 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:09.428513 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls podName:1b832eb9-54df-42b4-8e08-a3f28c1f598e nodeName:}" failed. No retries permitted until 2026-09-17 15:57:11.428505199 +0000 UTC m=+88.531750559 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6fskk" (UID: "1b832eb9-54df-42b4-8e08-a3f28c1f598e") : secret "insights-runtime-extractor-tls" not found Sep 17 15:57:09.556799 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:09.556769 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fj294_3c7b711c-7da3-4f63-ade9-9282296f6086/dns-node-resolver/0.log" Sep 17 15:57:10.147469 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:10.147431 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:10.755840 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:10.755817 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vlcft_edf5c407-8316-462e-b452-f867f1f5f301/node-ca/0.log" Sep 17 15:57:10.861998 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:10.861961 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:10.991458 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:10.991426 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:11.356328 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:11.356299 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-88zv4_5f94a7e8-2cf1-4051-a1d5-c7d29241fab3/serve-healthcheck-canary/0.log" Sep 17 15:57:11.448453 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:11.448410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:11.448647 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:11.448533 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:11.448647 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:11.448557 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 15:57:11.448647 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:11.448615 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 15:57:11.448647 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:11.448634 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls podName:1b832eb9-54df-42b4-8e08-a3f28c1f598e nodeName:}" failed. No retries permitted until 2026-09-17 15:57:15.448619324 +0000 UTC m=+92.551864695 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6fskk" (UID: "1b832eb9-54df-42b4-8e08-a3f28c1f598e") : secret "insights-runtime-extractor-tls" not found Sep 17 15:57:11.448806 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:11.448665 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm podName:1b832eb9-54df-42b4-8e08-a3f28c1f598e nodeName:}" failed. No retries permitted until 2026-09-17 15:57:15.448649242 +0000 UTC m=+92.551894619 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6fskk" (UID: "1b832eb9-54df-42b4-8e08-a3f28c1f598e") : configmap "kube-rbac-proxy" not found Sep 17 15:57:12.706352 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:12.706326 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7zb8p" Sep 17 15:57:13.029819 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:13.029729 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:13.132106 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:13.132071 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:57:13.132315 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:13.132133 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:57:13.132551 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:13.132535 2580 scope.go:117] "RemoveContainer" containerID="4d3c8231d51ae89526e729b570a19bbf305db9112f03629bb87bede7dfb36658" Sep 17 15:57:13.132745 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:13.132723 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-fd4bb46c-qp4q9_openshift-console-operator(0a3b84c4-ac8d-4e8a-a606-aba79b7fe148)\"" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" podUID="0a3b84c4-ac8d-4e8a-a606-aba79b7fe148" Sep 17 15:57:13.151327 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:13.151295 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:13.247695 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:13.247661 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:13.348278 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:13.348245 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:13.505551 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:13.505515 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:13.719209 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:13.719129 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:14.254442 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:14.254404 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:14.706394 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:14.706364 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:14.799354 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:14.799317 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:14.904673 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:14.904642 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:15.116276 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:15.116246 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:15.317963 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:15.317937 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:15.485473 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:15.485386 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:15.485473 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:15.485461 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 17 15:57:15.485655 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:15.485537 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm podName:1b832eb9-54df-42b4-8e08-a3f28c1f598e nodeName:}" failed. No retries permitted until 2026-09-17 15:57:23.485518287 +0000 UTC m=+100.588763658 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6fskk" (UID: "1b832eb9-54df-42b4-8e08-a3f28c1f598e") : configmap "kube-rbac-proxy" not found Sep 17 15:57:15.485655 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:15.485557 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:15.485655 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:15.485640 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 17 15:57:15.485754 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:15.485694 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls podName:1b832eb9-54df-42b4-8e08-a3f28c1f598e nodeName:}" failed. No retries permitted until 2026-09-17 15:57:23.485681961 +0000 UTC m=+100.588927321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6fskk" (UID: "1b832eb9-54df-42b4-8e08-a3f28c1f598e") : secret "insights-runtime-extractor-tls" not found Sep 17 15:57:15.521351 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:15.521315 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:15.910946 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:15.910912 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:16.491012 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:16.490981 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:16.844183 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:16.844153 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:16.932379 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:16.932346 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:17.088887 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:17.088860 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:17.493440 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:17.493412 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:17.583087 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:17.583057 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:18.282373 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:18.282338 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:18.680490 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:18.680460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:18.887969 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:18.887937 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:19.098690 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:19.098658 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:19.278030 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:19.278000 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:19.394802 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:19.394731 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:20.084233 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:20.084192 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:20.478495 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:20.478412 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:20.692658 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:20.692626 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:21.076934 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:21.076886 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:21.682773 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:21.682739 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:21.806124 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:21.806085 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:21.925679 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:21.925634 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:22.307608 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:22.307577 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:22.495640 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:22.495605 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:22.681142 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:22.681064 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:23.074959 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:23.074932 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:23.493889 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:23.493815 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:23.564684 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:23.564649 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:23.564837 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:23.564763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:23.565195 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:23.565172 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1b832eb9-54df-42b4-8e08-a3f28c1f598e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:23.566869 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:23.566848 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1b832eb9-54df-42b4-8e08-a3f28c1f598e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6fskk\" (UID: \"1b832eb9-54df-42b4-8e08-a3f28c1f598e\") " pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:23.570843 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:23.570824 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6fskk" Sep 17 15:57:23.686707 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:23.686672 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:24.023429 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.023391 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6fskk"] Sep 17 15:57:24.026569 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:57:24.026542 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b832eb9_54df_42b4_8e08_a3f28c1f598e.slice/crio-a16ebec7aec403c77dcbb93e0e48e70dab1885983970f65fc616c1827852f180 WatchSource:0}: Error finding container a16ebec7aec403c77dcbb93e0e48e70dab1885983970f65fc616c1827852f180: Status 404 returned error can't find the container with id a16ebec7aec403c77dcbb93e0e48e70dab1885983970f65fc616c1827852f180 Sep 17 15:57:24.137165 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.137136 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:24.334147 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.334120 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:24.572415 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.572378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:57:24.572586 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.572482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:57:24.574717 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.574689 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/9ac10347-81ed-4b23-8eac-ddc2a4c1d171-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-847b8bd8d7-5fpms\" (UID: \"9ac10347-81ed-4b23-8eac-ddc2a4c1d171\") " pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:57:24.574862 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.574838 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls\") pod \"image-registry-6f94b94954-h5n4x\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:57:24.672932 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.672900 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ghjkd\" (UID: \"51b96393-ed39-45b3-baca-b0acb42c254b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:57:24.673046 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.672952 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-9f49r\" (UID: \"7ad114b5-1283-442f-b7b2-fde01ae0f839\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:57:24.673046 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.672994 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:24.673046 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.673028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:57:24.673208 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.673112 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:24.673779 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.673701 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99684b4e-9484-4867-a730-ea1662218945-service-ca-bundle\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:24.675651 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.675609 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d24009de-229f-4bb3-a49e-5499cfc0b519-metrics-tls\") pod \"dns-default-2bdcw\" (UID: \"d24009de-229f-4bb3-a49e-5499cfc0b519\") " pod="openshift-dns/dns-default-2bdcw" Sep 17 15:57:24.676029 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.675996 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/7ad114b5-1283-442f-b7b2-fde01ae0f839-samples-operator-tls\") pod \"cluster-samples-operator-685d7d68b5-9f49r\" (UID: \"7ad114b5-1283-442f-b7b2-fde01ae0f839\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:57:24.676232 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.676198 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51b96393-ed39-45b3-baca-b0acb42c254b-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-ghjkd\" (UID: \"51b96393-ed39-45b3-baca-b0acb42c254b\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:57:24.676335 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.676289 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99684b4e-9484-4867-a730-ea1662218945-metrics-certs\") pod \"router-default-86dfcf9bc6-7j9bp\" (UID: \"99684b4e-9484-4867-a730-ea1662218945\") " pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:24.819824 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.819776 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" Sep 17 15:57:24.825956 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.825933 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:57:24.851141 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.851052 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" Sep 17 15:57:24.860270 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.860108 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:24.866204 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.866170 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" Sep 17 15:57:24.937202 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:24.937139 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2bdcw" Sep 17 15:57:25.000434 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:25.000273 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6fskk" event={"ID":"1b832eb9-54df-42b4-8e08-a3f28c1f598e","Type":"ContainerStarted","Data":"beecd727ff9c6ea0dfedd65df1e52fa778ec22bd5c8e86687b282d2c21e3c9fd"} Sep 17 15:57:25.000434 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:25.000378 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6fskk" event={"ID":"1b832eb9-54df-42b4-8e08-a3f28c1f598e","Type":"ContainerStarted","Data":"dec51ab007fc3250fa68a125b82467959342e70141d1a2d95972c069d996331e"} Sep 17 15:57:25.000434 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:25.000404 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6fskk" event={"ID":"1b832eb9-54df-42b4-8e08-a3f28c1f598e","Type":"ContainerStarted","Data":"a16ebec7aec403c77dcbb93e0e48e70dab1885983970f65fc616c1827852f180"} Sep 17 15:57:25.107577 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:25.107544 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms"] Sep 17 15:57:25.111591 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:57:25.111541 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac10347_81ed_4b23_8eac_ddc2a4c1d171.slice/crio-babecbb51b8b579ce4461dc966a7e5b7a98c124cfbe180a5a47b0e30a19e80fe WatchSource:0}: Error finding container babecbb51b8b579ce4461dc966a7e5b7a98c124cfbe180a5a47b0e30a19e80fe: Status 404 returned error can't find the container with id babecbb51b8b579ce4461dc966a7e5b7a98c124cfbe180a5a47b0e30a19e80fe Sep 17 15:57:25.127139 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:25.127108 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f94b94954-h5n4x"] Sep 17 15:57:25.128950 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:57:25.128924 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a11d802_951f_4a33_bb26_7b2d2b641ca6.slice/crio-11e1c34460e7ef7b8cf01462f816e8f043e01cf2478f93b41aaf7a5600b72362 WatchSource:0}: Error finding container 11e1c34460e7ef7b8cf01462f816e8f043e01cf2478f93b41aaf7a5600b72362: Status 404 returned error can't find the container with id 11e1c34460e7ef7b8cf01462f816e8f043e01cf2478f93b41aaf7a5600b72362 Sep 17 15:57:25.159560 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:25.159526 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd"] Sep 17 15:57:25.160928 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:25.160899 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r"] Sep 17 15:57:25.162744 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:57:25.162715 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51b96393_ed39_45b3_baca_b0acb42c254b.slice/crio-d39d4ab420648bc7e8a81fed38c60f772d34ffd61bcdda4fad88487cbe45c62b WatchSource:0}: Error finding container d39d4ab420648bc7e8a81fed38c60f772d34ffd61bcdda4fad88487cbe45c62b: Status 404 returned error can't find the container with id d39d4ab420648bc7e8a81fed38c60f772d34ffd61bcdda4fad88487cbe45c62b Sep 17 15:57:25.166046 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:25.165443 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-86dfcf9bc6-7j9bp"] Sep 17 15:57:25.174317 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:57:25.174292 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99684b4e_9484_4867_a730_ea1662218945.slice/crio-3a53fe1ed7499949d9c79c871f3f997315005ac4aa778f22498a4c5fcfb07977 WatchSource:0}: Error finding container 3a53fe1ed7499949d9c79c871f3f997315005ac4aa778f22498a4c5fcfb07977: Status 404 returned error can't find the container with id 3a53fe1ed7499949d9c79c871f3f997315005ac4aa778f22498a4c5fcfb07977 Sep 17 15:57:25.226149 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:25.226071 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2bdcw"] Sep 17 15:57:25.229905 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:57:25.229876 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd24009de_229f_4bb3_a49e_5499cfc0b519.slice/crio-2bb119f66506c19907afea0983c2ed643a71fca9e67550f7ed5f230545a54ab7 WatchSource:0}: Error finding container 2bb119f66506c19907afea0983c2ed643a71fca9e67550f7ed5f230545a54ab7: Status 404 returned error can't find the container with id 2bb119f66506c19907afea0983c2ed643a71fca9e67550f7ed5f230545a54ab7 Sep 17 15:57:26.007572 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.007535 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" event={"ID":"7ad114b5-1283-442f-b7b2-fde01ae0f839","Type":"ContainerStarted","Data":"a44b453346c41f23fc47899854915b5c52b70293261ff0d5ad0e0b46cea76f99"} Sep 17 15:57:26.008775 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.008749 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2bdcw" event={"ID":"d24009de-229f-4bb3-a49e-5499cfc0b519","Type":"ContainerStarted","Data":"2bb119f66506c19907afea0983c2ed643a71fca9e67550f7ed5f230545a54ab7"} Sep 17 15:57:26.010310 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.010285 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" event={"ID":"99684b4e-9484-4867-a730-ea1662218945","Type":"ContainerStarted","Data":"d247128849da8c0af9339ecbdb863ae9733d275bc5b552f97ebf4e7c9c0a24fd"} Sep 17 15:57:26.010415 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.010334 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" event={"ID":"99684b4e-9484-4867-a730-ea1662218945","Type":"ContainerStarted","Data":"3a53fe1ed7499949d9c79c871f3f997315005ac4aa778f22498a4c5fcfb07977"} Sep 17 15:57:26.011802 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.011779 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" event={"ID":"0a11d802-951f-4a33-bb26-7b2d2b641ca6","Type":"ContainerStarted","Data":"e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1"} Sep 17 15:57:26.011908 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.011817 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" event={"ID":"0a11d802-951f-4a33-bb26-7b2d2b641ca6","Type":"ContainerStarted","Data":"11e1c34460e7ef7b8cf01462f816e8f043e01cf2478f93b41aaf7a5600b72362"} Sep 17 15:57:26.011908 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.011846 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:57:26.013024 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.013001 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" event={"ID":"9ac10347-81ed-4b23-8eac-ddc2a4c1d171","Type":"ContainerStarted","Data":"babecbb51b8b579ce4461dc966a7e5b7a98c124cfbe180a5a47b0e30a19e80fe"} Sep 17 15:57:26.014352 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.014326 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" event={"ID":"51b96393-ed39-45b3-baca-b0acb42c254b","Type":"ContainerStarted","Data":"d39d4ab420648bc7e8a81fed38c60f772d34ffd61bcdda4fad88487cbe45c62b"} Sep 17 15:57:26.037748 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.037689 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" podStartSLOduration=116.037669306 podStartE2EDuration="1m56.037669306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:26.036451497 +0000 UTC m=+103.139696909" watchObservedRunningTime="2026-09-17 15:57:26.037669306 +0000 UTC m=+103.140914679" Sep 17 15:57:26.488341 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.488309 2580 scope.go:117] "RemoveContainer" containerID="4d3c8231d51ae89526e729b570a19bbf305db9112f03629bb87bede7dfb36658" Sep 17 15:57:26.860374 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.860340 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:26.863702 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.863675 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:26.888527 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:26.888466 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" podStartSLOduration=119.888442068 podStartE2EDuration="1m59.888442068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:26.065888114 +0000 UTC m=+103.169133497" watchObservedRunningTime="2026-09-17 15:57:26.888442068 +0000 UTC m=+103.991687451" Sep 17 15:57:27.020428 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:27.020390 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6fskk" event={"ID":"1b832eb9-54df-42b4-8e08-a3f28c1f598e","Type":"ContainerStarted","Data":"5c64d60f1acca8172184947763a4ce8e282c56e1f9e19321c9c556d194c13bc8"} Sep 17 15:57:27.020750 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:27.020715 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:27.022056 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:27.022035 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-86dfcf9bc6-7j9bp" Sep 17 15:57:27.039626 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:27.039581 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6fskk" podStartSLOduration=18.217613689 podStartE2EDuration="20.039564106s" totalImagesPullingTime="1.821950417s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:57:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:27.039436858 +0000 UTC m=+104.142682240" watchObservedRunningTime="2026-09-17 15:57:27.039564106 +0000 UTC m=+104.142809491" Sep 17 15:57:28.508372 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:28.508331 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:28.614308 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:28.614230 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:28.707866 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:28.707832 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:28.894962 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:28.894887 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:29.003522 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.003491 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:29.028870 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.028838 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" event={"ID":"51b96393-ed39-45b3-baca-b0acb42c254b","Type":"ContainerStarted","Data":"47572784c4a1202003a375dd596d0b265f5ab60fdde1ec3c7a682a2bf6fa8333"} Sep 17 15:57:29.030806 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.030774 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" event={"ID":"7ad114b5-1283-442f-b7b2-fde01ae0f839","Type":"ContainerStarted","Data":"6735881bca7ff13a207bf30ff728a552221248c07362de49238dfdb2d1fd2a6c"} Sep 17 15:57:29.030943 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.030819 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" event={"ID":"7ad114b5-1283-442f-b7b2-fde01ae0f839","Type":"ContainerStarted","Data":"89bb2e3ce8f1576cb8e52725b161a515181beefd864a8e01698074057b89e9fc"} Sep 17 15:57:29.032598 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.032568 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2bdcw" event={"ID":"d24009de-229f-4bb3-a49e-5499cfc0b519","Type":"ContainerStarted","Data":"5e860c7cab9553ad518186bfb118304a998d72c87f67baad429bf13b36119df6"} Sep 17 15:57:29.032722 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.032611 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2bdcw" event={"ID":"d24009de-229f-4bb3-a49e-5499cfc0b519","Type":"ContainerStarted","Data":"2954b7c10f14a367aeec6d5bac05ffd1b6c4227b698771e815b7dcd36082f151"} Sep 17 15:57:29.032722 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.032638 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2bdcw" Sep 17 15:57:29.034025 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.034000 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" event={"ID":"9ac10347-81ed-4b23-8eac-ddc2a4c1d171","Type":"ContainerStarted","Data":"8e19b6e865475b52b838ab94724ae8071a1cc9756894b5418beb0b08823fd2b7"} Sep 17 15:57:29.035874 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.035855 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 15:57:29.035978 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.035960 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" event={"ID":"0a3b84c4-ac8d-4e8a-a606-aba79b7fe148","Type":"ContainerStarted","Data":"fd72494c4dd36db45cd8dd7acf5d145ac671ee125bbeb33c854530b2891425eb"} Sep 17 15:57:29.036342 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.036321 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:57:29.040970 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.040951 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" Sep 17 15:57:29.048641 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.048600 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-ghjkd" podStartSLOduration=88.142042599 podStartE2EDuration="1m31.048586653s" totalImagesPullingTime="2.906544054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:29.047516777 +0000 UTC m=+106.150762158" watchObservedRunningTime="2026-09-17 15:57:29.048586653 +0000 UTC m=+106.151832034" Sep 17 15:57:29.068754 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.068708 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-847b8bd8d7-5fpms" podStartSLOduration=116.109149235 podStartE2EDuration="1m59.06869603s" totalImagesPullingTime="2.959546795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:29.067595291 +0000 UTC m=+106.170840671" watchObservedRunningTime="2026-09-17 15:57:29.06869603 +0000 UTC m=+106.171941413" Sep 17 15:57:29.088034 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.087996 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2bdcw" podStartSLOduration=34.246549438 podStartE2EDuration="37.087983672s" totalImagesPullingTime="2.841434234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:29.086124458 +0000 UTC m=+106.189369859" watchObservedRunningTime="2026-09-17 15:57:29.087983672 +0000 UTC m=+106.191229053" Sep 17 15:57:29.094567 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.094546 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:29.106976 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.106942 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-685d7d68b5-9f49r" podStartSLOduration=116.233002009 podStartE2EDuration="1m59.106931615s" totalImagesPullingTime="2.873929606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:29.106837429 +0000 UTC m=+106.210082810" watchObservedRunningTime="2026-09-17 15:57:29.106931615 +0000 UTC m=+106.210176996" Sep 17 15:57:29.125202 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.125169 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-fd4bb46c-qp4q9" podStartSLOduration=107.341935058 podStartE2EDuration="1m59.125157793s" totalImagesPullingTime="11.783222735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:29.124363364 +0000 UTC m=+106.227608740" watchObservedRunningTime="2026-09-17 15:57:29.125157793 +0000 UTC m=+106.228403174" Sep 17 15:57:29.201277 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.197955 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:29.568504 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.568472 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:29.645891 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.645866 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:29.756838 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.756808 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:29.857610 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.857524 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:29.957153 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:29.957118 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:30.031071 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:30.031047 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:30.564529 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:30.564499 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:30.775848 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:30.775815 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:30.968077 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:30.967994 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:31.805490 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:31.805454 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:31.905056 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:31.905024 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:32.170100 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:32.170020 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:32.911416 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:32.911388 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:33.015401 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:33.015315 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:33.131003 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:33.130972 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:33.245382 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:33.245348 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:33.377561 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:33.377534 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:35.721907 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:35.721876 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:35.853426 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:35.853390 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:36.281740 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:36.281710 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:39.041834 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:39.041805 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2bdcw" Sep 17 15:57:39.227637 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:39.227606 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:39.658386 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:39.658352 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:39.735732 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:39.735701 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:40.907874 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:40.907835 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:41.136976 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.136944 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:41.197996 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.197911 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fp6xb"] Sep 17 15:57:41.201092 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.201067 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.204087 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.204063 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t8vk7\"" Sep 17 15:57:41.204206 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.204138 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 17 15:57:41.204302 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.204293 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 17 15:57:41.205207 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.205179 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 17 15:57:41.205207 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.205191 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 17 15:57:41.228409 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.228388 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:41.320187 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.320157 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:41.322290 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.322262 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.322432 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.322309 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-tls\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.322432 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.322340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b037dd9f-6811-40ad-9077-b085f2046267-root\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.322432 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.322362 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b037dd9f-6811-40ad-9077-b085f2046267-metrics-client-ca\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.322593 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.322463 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chl95\" (UniqueName: \"kubernetes.io/projected/b037dd9f-6811-40ad-9077-b085f2046267-kube-api-access-chl95\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.322593 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.322501 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-wtmp\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.322593 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.322535 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-accelerators-collector-config\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.322593 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.322562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b037dd9f-6811-40ad-9077-b085f2046267-sys\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.322593 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.322584 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-textfile\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.419425 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.419392 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:41.423415 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.423393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b037dd9f-6811-40ad-9077-b085f2046267-root\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.423534 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.423427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b037dd9f-6811-40ad-9077-b085f2046267-metrics-client-ca\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.423534 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.423494 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b037dd9f-6811-40ad-9077-b085f2046267-root\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.423642 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.423552 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chl95\" (UniqueName: \"kubernetes.io/projected/b037dd9f-6811-40ad-9077-b085f2046267-kube-api-access-chl95\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.423642 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.423571 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-wtmp\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.423642 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.423591 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-accelerators-collector-config\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.423642 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.423611 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b037dd9f-6811-40ad-9077-b085f2046267-sys\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.423642 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.423627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-textfile\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.423894 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.423701 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.423894 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.423709 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b037dd9f-6811-40ad-9077-b085f2046267-sys\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.423894 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.423731 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-wtmp\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.423894 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.423834 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-tls\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.424090 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:41.423951 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 17 15:57:41.424090 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.424018 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-textfile\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.424090 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:57:41.424026 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-tls podName:b037dd9f-6811-40ad-9077-b085f2046267 nodeName:}" failed. No retries permitted until 2026-09-17 15:57:41.924009032 +0000 UTC m=+119.027254411 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-tls") pod "node-exporter-fp6xb" (UID: "b037dd9f-6811-40ad-9077-b085f2046267") : secret "node-exporter-tls" not found Sep 17 15:57:41.424090 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.424073 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b037dd9f-6811-40ad-9077-b085f2046267-metrics-client-ca\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.424272 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.424142 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-accelerators-collector-config\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.426054 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.426038 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.434678 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.434657 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chl95\" (UniqueName: \"kubernetes.io/projected/b037dd9f-6811-40ad-9077-b085f2046267-kube-api-access-chl95\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.520853 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.520790 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:41.620356 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.620317 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:41.796916 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.796879 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:41.927596 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.927565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-tls\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.929874 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.929848 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b037dd9f-6811-40ad-9077-b085f2046267-node-exporter-tls\") pod \"node-exporter-fp6xb\" (UID: \"b037dd9f-6811-40ad-9077-b085f2046267\") " pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:41.948096 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:41.948067 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:42.111283 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:42.111176 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fp6xb" Sep 17 15:57:42.122252 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:57:42.122206 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb037dd9f_6811_40ad_9077_b085f2046267.slice/crio-c1dc16916c96462d2cdb210e5511377853272a632b3356d9cc8d5424ea607f6e WatchSource:0}: Error finding container c1dc16916c96462d2cdb210e5511377853272a632b3356d9cc8d5424ea607f6e: Status 404 returned error can't find the container with id c1dc16916c96462d2cdb210e5511377853272a632b3356d9cc8d5424ea607f6e Sep 17 15:57:42.150468 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:42.150441 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:42.252212 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:42.252181 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:42.369885 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:42.369809 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:43.089125 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:43.089091 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fp6xb" event={"ID":"b037dd9f-6811-40ad-9077-b085f2046267","Type":"ContainerStarted","Data":"3fdd18634be8380ea987477523a0dabcd413d899d8859b80da8015234bb67021"} Sep 17 15:57:43.089428 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:43.089170 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fp6xb" event={"ID":"b037dd9f-6811-40ad-9077-b085f2046267","Type":"ContainerStarted","Data":"c1dc16916c96462d2cdb210e5511377853272a632b3356d9cc8d5424ea607f6e"} Sep 17 15:57:43.995257 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:43.995201 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:44.094920 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:44.094890 2580 generic.go:356] "Generic (PLEG): container finished" podID="b037dd9f-6811-40ad-9077-b085f2046267" containerID="3fdd18634be8380ea987477523a0dabcd413d899d8859b80da8015234bb67021" exitCode=0 Sep 17 15:57:44.095340 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:44.094929 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fp6xb" event={"ID":"b037dd9f-6811-40ad-9077-b085f2046267","Type":"ContainerDied","Data":"3fdd18634be8380ea987477523a0dabcd413d899d8859b80da8015234bb67021"} Sep 17 15:57:44.095986 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:44.095955 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:44.197021 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:44.196968 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:44.295619 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:44.295590 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:44.414857 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:44.414829 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:44.830349 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:44.830318 2580 patch_prober.go:28] interesting pod/image-registry-6f94b94954-h5n4x container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 17 15:57:44.830503 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:44.830375 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" podUID="0a11d802-951f-4a33-bb26-7b2d2b641ca6" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 15:57:45.100871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:45.100788 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fp6xb" event={"ID":"b037dd9f-6811-40ad-9077-b085f2046267","Type":"ContainerStarted","Data":"3c230a514a4e55040642ce7636b64713b50df67d25e1fd73a843edfe5006eb19"} Sep 17 15:57:45.100871 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:45.100852 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fp6xb" event={"ID":"b037dd9f-6811-40ad-9077-b085f2046267","Type":"ContainerStarted","Data":"692c4c1b328e99859e4e69a7ab03ccf4bc77931da97e4f584f33ba3df1953a15"} Sep 17 15:57:45.119947 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:45.119897 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fp6xb" podStartSLOduration=3.164610902 podStartE2EDuration="4.119882413s" totalImagesPullingTime="890.480098ms" totalInitContainerRuntime="64.791413ms" isStatefulPod=true podCreationTimestamp="2026-09-17 15:57:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:45.118584178 +0000 UTC m=+122.221829560" watchObservedRunningTime="2026-09-17 15:57:45.119882413 +0000 UTC m=+122.223127837" Sep 17 15:57:45.911761 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:45.911721 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:46.006564 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:46.006531 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:46.342427 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:46.342399 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:46.634342 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:46.634272 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:46.734844 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:46.734796 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:47.025300 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:47.025206 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:57:47.415325 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:47.415289 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:47.509984 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:47.509937 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:50.283128 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:50.283097 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:50.444946 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:50.444910 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:50.547849 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:50.547824 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:50.787334 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:50.787312 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:50.897684 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:50.897612 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:51.606123 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:51.606067 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:51.680182 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:51.680153 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:52.114474 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.114439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:57:52.114639 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.114483 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:57:52.120116 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.120099 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 17 15:57:52.120779 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.120763 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 17 15:57:52.126950 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.126926 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/609285b3-0b12-4a2c-a723-afb1861e75a4-original-pull-secret\") pod \"global-pull-secret-syncer-8877v\" (UID: \"609285b3-0b12-4a2c-a723-afb1861e75a4\") " pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:57:52.127100 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.127077 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05ed94c8-e908-40bc-abf2-b770ebc1344a-metrics-certs\") pod \"network-metrics-daemon-8clfl\" (UID: \"05ed94c8-e908-40bc-abf2-b770ebc1344a\") " pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:57:52.214981 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.214951 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdfhg\" (UniqueName: \"kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg\") pod \"network-check-target-kkn7c\" (UID: \"af1e77ed-0bbf-419b-8a67-029c77b67873\") " pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:57:52.217277 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.217248 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdfhg\" (UniqueName: \"kubernetes.io/projected/af1e77ed-0bbf-419b-8a67-029c77b67873-kube-api-access-vdfhg\") pod \"network-check-target-kkn7c\" (UID: \"af1e77ed-0bbf-419b-8a67-029c77b67873\") " pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:57:52.414718 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.414653 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fvxq9\"" Sep 17 15:57:52.416741 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.416727 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:57:52.417844 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.417828 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8877v" Sep 17 15:57:52.431384 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.431362 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-v87nd\"" Sep 17 15:57:52.436643 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.436623 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8clfl" Sep 17 15:57:52.692907 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:57:52.692875 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod609285b3_0b12_4a2c_a723_afb1861e75a4.slice/crio-518c9a1eede7b943281942aea507432802e0db9f64dd1f714d3e076d56ef51fb WatchSource:0}: Error finding container 518c9a1eede7b943281942aea507432802e0db9f64dd1f714d3e076d56ef51fb: Status 404 returned error can't find the container with id 518c9a1eede7b943281942aea507432802e0db9f64dd1f714d3e076d56ef51fb Sep 17 15:57:52.693407 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.693376 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8877v"] Sep 17 15:57:52.694711 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.694691 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kkn7c"] Sep 17 15:57:52.697717 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:57:52.697696 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf1e77ed_0bbf_419b_8a67_029c77b67873.slice/crio-ccb8eee04995e3c454244073e7948eb21e79ad54063b60338dd7c5f3b965a8b9 WatchSource:0}: Error finding container ccb8eee04995e3c454244073e7948eb21e79ad54063b60338dd7c5f3b965a8b9: Status 404 returned error can't find the container with id ccb8eee04995e3c454244073e7948eb21e79ad54063b60338dd7c5f3b965a8b9 Sep 17 15:57:52.724095 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:52.724078 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8clfl"] Sep 17 15:57:52.726163 ip-10-0-128-222 kubenswrapper[2580]: W0917 15:57:52.726142 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ed94c8_e908_40bc_abf2_b770ebc1344a.slice/crio-d6224995b3b021fe29b1a77843e10d9a5c32833ccebe8a29e73fb7523158b328 WatchSource:0}: Error finding container d6224995b3b021fe29b1a77843e10d9a5c32833ccebe8a29e73fb7523158b328: Status 404 returned error can't find the container with id d6224995b3b021fe29b1a77843e10d9a5c32833ccebe8a29e73fb7523158b328 Sep 17 15:57:53.140826 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:53.140694 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kkn7c" event={"ID":"af1e77ed-0bbf-419b-8a67-029c77b67873","Type":"ContainerStarted","Data":"6fd64657351da83ff81831f2e2cbee859e0f225c6562c1406dd1baa06412e2df"} Sep 17 15:57:53.140826 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:53.140761 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kkn7c" event={"ID":"af1e77ed-0bbf-419b-8a67-029c77b67873","Type":"ContainerStarted","Data":"ccb8eee04995e3c454244073e7948eb21e79ad54063b60338dd7c5f3b965a8b9"} Sep 17 15:57:53.140826 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:53.140793 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:57:53.142729 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:53.142698 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8877v" event={"ID":"609285b3-0b12-4a2c-a723-afb1861e75a4","Type":"ContainerStarted","Data":"518c9a1eede7b943281942aea507432802e0db9f64dd1f714d3e076d56ef51fb"} Sep 17 15:57:53.144537 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:53.144462 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8clfl" event={"ID":"05ed94c8-e908-40bc-abf2-b770ebc1344a","Type":"ContainerStarted","Data":"d6224995b3b021fe29b1a77843e10d9a5c32833ccebe8a29e73fb7523158b328"} Sep 17 15:57:53.896047 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:53.895999 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:53.999471 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:53.999395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:54.151889 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:54.151562 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8clfl" event={"ID":"05ed94c8-e908-40bc-abf2-b770ebc1344a","Type":"ContainerStarted","Data":"3930f2b14e84a2fd3c451592d18fba674bec9ebe314078b9c1b57095464d85f9"} Sep 17 15:57:54.151889 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:54.151622 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8clfl" event={"ID":"05ed94c8-e908-40bc-abf2-b770ebc1344a","Type":"ContainerStarted","Data":"c9c4c836681e54e94dd1b0ed5651e178e53c29004d1a92253a73c627ca4a64ac"} Sep 17 15:57:54.168165 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:54.168119 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8clfl" podStartSLOduration=130.210783005 podStartE2EDuration="2m11.168105155s" totalImagesPullingTime="957.32215ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:54.166953995 +0000 UTC m=+131.270199380" watchObservedRunningTime="2026-09-17 15:57:54.168105155 +0000 UTC m=+131.271350537" Sep 17 15:57:54.168782 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:54.168734 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kkn7c" podStartSLOduration=131.16871896 podStartE2EDuration="2m11.16871896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 15:55:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:53.16380631 +0000 UTC m=+130.267051693" watchObservedRunningTime="2026-09-17 15:57:54.16871896 +0000 UTC m=+131.271964347" Sep 17 15:57:54.293646 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:54.293610 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:54.393536 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:54.393491 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:54.467967 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:54.467935 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:54.921842 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:54.921810 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f94b94954-h5n4x"] Sep 17 15:57:55.062447 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:55.062413 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:55.163721 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:55.163677 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:55.634150 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:55.634113 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:56.606446 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:56.606411 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:56.706911 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:56.706880 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:57.165607 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:57.165574 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8877v" event={"ID":"609285b3-0b12-4a2c-a723-afb1861e75a4","Type":"ContainerStarted","Data":"923b4ab80e20b62a2ef76aab5c5af2609ee223588dc439984958e5a0e22cfbb1"} Sep 17 15:57:57.181704 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:57.181662 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8877v" podStartSLOduration=130.391222359 podStartE2EDuration="2m14.181642476s" totalImagesPullingTime="3.790420117s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 15:55:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 15:57:57.180207884 +0000 UTC m=+134.283453326" watchObservedRunningTime="2026-09-17 15:57:57.181642476 +0000 UTC m=+134.284887859" Sep 17 15:57:57.185266 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:57.185238 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:57.287199 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:57.287179 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:57.853894 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:57.853866 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:57:57.953152 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:57:57.953126 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:00.693560 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:00.693530 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:00.783092 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:00.783056 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:03.122000 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:03.121970 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:03.189173 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:03.189142 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:08.175782 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:08.175728 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:08.278019 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:08.277965 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:08.346096 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:08.346062 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:08.848884 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:08.848849 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:08.951638 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:08.951607 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:09.029275 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:09.029206 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:19.945456 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:19.945399 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" podUID="0a11d802-951f-4a33-bb26-7b2d2b641ca6" containerName="registry" containerID="cri-o://e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1" gracePeriod=30 Sep 17 15:58:20.180018 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.179992 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:58:20.226886 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.226812 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a11d802-951f-4a33-bb26-7b2d2b641ca6-installation-pull-secrets\") pod \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " Sep 17 15:58:20.226886 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.226864 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67bn9\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-kube-api-access-67bn9\") pod \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " Sep 17 15:58:20.227092 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.226896 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a11d802-951f-4a33-bb26-7b2d2b641ca6-trusted-ca\") pod \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " Sep 17 15:58:20.227092 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.227021 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls\") pod \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " Sep 17 15:58:20.227092 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.227067 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a11d802-951f-4a33-bb26-7b2d2b641ca6-image-registry-private-configuration\") pod \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " Sep 17 15:58:20.227266 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.227107 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a11d802-951f-4a33-bb26-7b2d2b641ca6-ca-trust-extracted\") pod \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " Sep 17 15:58:20.227266 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.227138 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-certificates\") pod \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " Sep 17 15:58:20.227266 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.227202 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-bound-sa-token\") pod \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\" (UID: \"0a11d802-951f-4a33-bb26-7b2d2b641ca6\") " Sep 17 15:58:20.227419 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.227378 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a11d802-951f-4a33-bb26-7b2d2b641ca6-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0a11d802-951f-4a33-bb26-7b2d2b641ca6" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 15:58:20.227502 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.227479 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0a11d802-951f-4a33-bb26-7b2d2b641ca6-trusted-ca\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 15:58:20.228190 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.228068 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0a11d802-951f-4a33-bb26-7b2d2b641ca6" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 15:58:20.229719 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.229691 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a11d802-951f-4a33-bb26-7b2d2b641ca6-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0a11d802-951f-4a33-bb26-7b2d2b641ca6" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 15:58:20.229842 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.229774 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-kube-api-access-67bn9" (OuterVolumeSpecName: "kube-api-access-67bn9") pod "0a11d802-951f-4a33-bb26-7b2d2b641ca6" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6"). InnerVolumeSpecName "kube-api-access-67bn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 15:58:20.229980 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.229943 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a11d802-951f-4a33-bb26-7b2d2b641ca6-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0a11d802-951f-4a33-bb26-7b2d2b641ca6" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 15:58:20.230102 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.229956 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0a11d802-951f-4a33-bb26-7b2d2b641ca6" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 15:58:20.230334 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.230310 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0a11d802-951f-4a33-bb26-7b2d2b641ca6" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 17 15:58:20.235923 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.235898 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a11d802-951f-4a33-bb26-7b2d2b641ca6-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0a11d802-951f-4a33-bb26-7b2d2b641ca6" (UID: "0a11d802-951f-4a33-bb26-7b2d2b641ca6"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 17 15:58:20.254340 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.254315 2580 generic.go:356] "Generic (PLEG): container finished" podID="0a11d802-951f-4a33-bb26-7b2d2b641ca6" containerID="e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1" exitCode=0 Sep 17 15:58:20.254431 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.254354 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" Sep 17 15:58:20.254431 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.254395 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" event={"ID":"0a11d802-951f-4a33-bb26-7b2d2b641ca6","Type":"ContainerDied","Data":"e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1"} Sep 17 15:58:20.254535 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.254454 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f94b94954-h5n4x" event={"ID":"0a11d802-951f-4a33-bb26-7b2d2b641ca6","Type":"ContainerDied","Data":"11e1c34460e7ef7b8cf01462f816e8f043e01cf2478f93b41aaf7a5600b72362"} Sep 17 15:58:20.254535 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.254483 2580 scope.go:117] "RemoveContainer" containerID="e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1" Sep 17 15:58:20.262776 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.262763 2580 scope.go:117] "RemoveContainer" containerID="e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1" Sep 17 15:58:20.263032 ip-10-0-128-222 kubenswrapper[2580]: E0917 15:58:20.263016 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1\": container with ID starting with e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1 not found: ID does not exist" containerID="e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1" Sep 17 15:58:20.263075 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.263040 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1"} err="failed to get container status \"e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1\": rpc error: code = NotFound desc = could not find container \"e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1\": container with ID starting with e7ff6497637dca51dd8a5f8614df52407313b47ced5d5d726f068554b682a1e1 not found: ID does not exist" Sep 17 15:58:20.278810 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.278790 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6f94b94954-h5n4x"] Sep 17 15:58:20.284976 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.284952 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6f94b94954-h5n4x"] Sep 17 15:58:20.328154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.328122 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-bound-sa-token\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 15:58:20.328154 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.328150 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0a11d802-951f-4a33-bb26-7b2d2b641ca6-installation-pull-secrets\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 15:58:20.328351 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.328164 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67bn9\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-kube-api-access-67bn9\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 15:58:20.328351 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.328177 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-tls\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 15:58:20.328351 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.328190 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0a11d802-951f-4a33-bb26-7b2d2b641ca6-image-registry-private-configuration\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 15:58:20.328351 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.328202 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0a11d802-951f-4a33-bb26-7b2d2b641ca6-ca-trust-extracted\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 15:58:20.328351 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:20.328228 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0a11d802-951f-4a33-bb26-7b2d2b641ca6-registry-certificates\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 15:58:21.492071 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:21.492042 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a11d802-951f-4a33-bb26-7b2d2b641ca6" path="/var/lib/kubelet/pods/0a11d802-951f-4a33-bb26-7b2d2b641ca6/volumes" Sep 17 15:58:22.263877 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:22.263847 2580 generic.go:356] "Generic (PLEG): container finished" podID="fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b" containerID="a7d1586a1402111884dbba0ae43c017cbf3bfec641f42f0baaa5c04a9fe41078" exitCode=0 Sep 17 15:58:22.264046 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:22.263914 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" event={"ID":"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b","Type":"ContainerDied","Data":"a7d1586a1402111884dbba0ae43c017cbf3bfec641f42f0baaa5c04a9fe41078"} Sep 17 15:58:22.264235 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:22.264207 2580 scope.go:117] "RemoveContainer" containerID="a7d1586a1402111884dbba0ae43c017cbf3bfec641f42f0baaa5c04a9fe41078" Sep 17 15:58:23.269493 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:23.269459 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-wps4t" event={"ID":"fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b","Type":"ContainerStarted","Data":"b4fe1114f108180ee55976cccfe6a70da961ab4c71ebc1f8ce69eaa2f431b695"} Sep 17 15:58:23.872370 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:23.872341 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:23.951394 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:23.951364 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:24.154399 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:24.154321 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kkn7c" Sep 17 15:58:26.282206 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:26.282174 2580 generic.go:356] "Generic (PLEG): container finished" podID="060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4" containerID="62708dfa5ca10dbceb2d4774e31d617d57533384e3c395a9c70d92b74a72acdd" exitCode=0 Sep 17 15:58:26.282542 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:26.282254 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" event={"ID":"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4","Type":"ContainerDied","Data":"62708dfa5ca10dbceb2d4774e31d617d57533384e3c395a9c70d92b74a72acdd"} Sep 17 15:58:26.282585 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:26.282571 2580 scope.go:117] "RemoveContainer" containerID="62708dfa5ca10dbceb2d4774e31d617d57533384e3c395a9c70d92b74a72acdd" Sep 17 15:58:27.287385 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:27.287353 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57cdc4b768-mmrtc" event={"ID":"060ea9fe-ba22-4ee2-a7dc-a5e6b46ab4d4","Type":"ContainerStarted","Data":"e43223509b515fbdf24059c1606c77b61615e3f17d204d09546ff432ead1e73a"} Sep 17 15:58:31.203117 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:31.203084 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:31.273054 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:31.273021 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:33.117020 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:33.116981 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" podUID="49f537d0-1508-448d-af52-bc6b8fe22ece" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 17 15:58:37.333270 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:37.333235 2580 generic.go:356] "Generic (PLEG): container finished" podID="4861c6ad-a4a3-4947-a1f1-b21601e7a455" containerID="0250c48096b52de0dd35eaad819cedde996b39342027dd61106f235db274a08b" exitCode=0 Sep 17 15:58:37.333628 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:37.333310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" event={"ID":"4861c6ad-a4a3-4947-a1f1-b21601e7a455","Type":"ContainerDied","Data":"0250c48096b52de0dd35eaad819cedde996b39342027dd61106f235db274a08b"} Sep 17 15:58:37.333628 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:37.333602 2580 scope.go:117] "RemoveContainer" containerID="0250c48096b52de0dd35eaad819cedde996b39342027dd61106f235db274a08b" Sep 17 15:58:38.339346 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:38.339299 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-lh7sh" event={"ID":"4861c6ad-a4a3-4947-a1f1-b21601e7a455","Type":"ContainerStarted","Data":"6e0da1433b05e89608dbef37390b33357070a09234ba6c953e5ec324aed12670"} Sep 17 15:58:43.117161 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:43.117120 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" podUID="49f537d0-1508-448d-af52-bc6b8fe22ece" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 17 15:58:53.116979 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:53.116938 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" podUID="49f537d0-1508-448d-af52-bc6b8fe22ece" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Sep 17 15:58:53.117498 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:53.117013 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" Sep 17 15:58:53.117639 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:53.117616 2580 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"9723bbe82e151537b674d76165cc52fb26169515ec6cd1283c092f427c997aa4"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" containerMessage="Container service-proxy failed liveness probe, will be restarted" Sep 17 15:58:53.117703 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:53.117670 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" podUID="49f537d0-1508-448d-af52-bc6b8fe22ece" containerName="service-proxy" containerID="cri-o://9723bbe82e151537b674d76165cc52fb26169515ec6cd1283c092f427c997aa4" gracePeriod=30 Sep 17 15:58:53.398097 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:53.398018 2580 generic.go:356] "Generic (PLEG): container finished" podID="49f537d0-1508-448d-af52-bc6b8fe22ece" containerID="9723bbe82e151537b674d76165cc52fb26169515ec6cd1283c092f427c997aa4" exitCode=2 Sep 17 15:58:53.398097 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:53.398082 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" event={"ID":"49f537d0-1508-448d-af52-bc6b8fe22ece","Type":"ContainerDied","Data":"9723bbe82e151537b674d76165cc52fb26169515ec6cd1283c092f427c997aa4"} Sep 17 15:58:53.398300 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:53.398115 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f4858d95-kxhhh" event={"ID":"49f537d0-1508-448d-af52-bc6b8fe22ece","Type":"ContainerStarted","Data":"87dc2004faf81f3f01f0af9e4999cb24eab49367438713c1e54f0df59bca35a4"} Sep 17 15:58:57.373581 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:57.373520 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:58:57.477096 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:58:57.477069 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:59:09.313821 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:59:09.313784 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:59:10.629001 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:59:10.628967 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:59:16.265372 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:59:16.265329 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:59:20.356314 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:59:20.356280 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:59:20.431804 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:59:20.431769 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:59:23.871132 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:59:23.871090 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:59:23.969774 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:59:23.969744 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:59:27.534750 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:59:27.534720 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:59:27.635421 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:59:27.635390 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:59:27.750531 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:59:27.750501 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:59:27.823736 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:59:27.823707 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 15:59:53.134383 ip-10-0-128-222 kubenswrapper[2580]: I0917 15:59:53.134347 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:00:21.562437 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:00:21.562351 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:00:42.788357 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:00:42.788320 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:00:43.240465 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:00:43.240437 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:00:43.252416 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:00:43.252385 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:00:43.255032 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:00:43.255006 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:00:43.258079 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:00:43.258054 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:00:43.260762 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:00:43.260743 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:00:43.262110 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:00:43.262092 2580 kubelet.go:1648] "Image garbage collection succeeded" Sep 17 16:00:43.263357 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:00:43.263343 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:00:44.591498 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:00:44.591462 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:00:44.679461 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:00:44.679431 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:06.055916 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:06.055880 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:10.994452 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:10.994413 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:11.092781 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:11.092749 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:19.119871 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:19.119829 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:19.477107 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:19.477026 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:19.667898 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:19.667865 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:20.142438 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:20.142401 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:20.241429 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:20.241395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:20.513028 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:20.512949 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:20.605098 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:20.605066 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:20.851335 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:20.851302 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:21.134291 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:21.134183 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:21.241834 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:21.241799 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:31.644282 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:31.644245 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:01:31.712334 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:01:31.712300 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:06.428290 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:06.428258 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:06.495259 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:06.495203 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:09.680060 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:09.680026 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:09.782195 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:09.782160 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:43.220750 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:43.220713 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:43.294297 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:43.294265 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:45.368763 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:45.368726 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:45.502533 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:45.502499 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:45.620309 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:45.620197 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:45.770411 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:45.770376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:45.946085 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:45.946004 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:46.327888 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:46.327856 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:46.435540 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:46.435508 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:46.527561 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:46.527528 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:54.429139 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:54.429106 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:54.532340 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:54.532306 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:54.641544 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:54.641508 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:02:58.697698 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:02:58.697615 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:03:35.264201 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:03:35.264124 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:04:08.725001 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:04:08.724961 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:04:08.825782 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:04:08.825744 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:04:08.952476 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:04:08.952443 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:04:09.055048 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:04:09.055014 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:04:09.162952 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:04:09.162924 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:04:10.523907 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:04:10.523873 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:04:10.614632 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:04:10.614603 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:05:43.277654 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:43.277617 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:05:43.280116 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:43.280089 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:05:43.286908 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:43.286886 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:05:43.288297 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:43.288279 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:05:43.289574 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:43.289558 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:05:43.290871 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:43.290841 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:05:44.926525 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:44.926492 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:05:45.011677 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:45.011643 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:05:57.650504 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:57.650469 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:05:57.960525 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:57.960441 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:05:58.166204 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:58.166158 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:05:58.261619 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:58.261529 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:05:58.350900 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:58.350865 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:05:58.463670 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:58.463633 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:05:58.621932 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:58.621901 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:05:58.704728 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:58.704691 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:05:59.080469 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:05:59.080434 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:06:19.571506 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:06:19.571469 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:06:48.733700 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:06:48.733663 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:07:37.794990 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.794911 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:07:37.842084 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.842035 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt"] Sep 17 16:07:37.842707 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.842682 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a11d802-951f-4a33-bb26-7b2d2b641ca6" containerName="registry" Sep 17 16:07:37.842707 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.842706 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a11d802-951f-4a33-bb26-7b2d2b641ca6" containerName="registry" Sep 17 16:07:37.842870 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.842846 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a11d802-951f-4a33-bb26-7b2d2b641ca6" containerName="registry" Sep 17 16:07:37.846298 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.846275 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:07:37.858507 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.858481 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt"] Sep 17 16:07:37.869727 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.869686 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-48b6f-serving-cert\"" Sep 17 16:07:37.869931 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.869785 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 17 16:07:37.869931 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.869785 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-48b6f-kube-rbac-proxy-sar-config\"" Sep 17 16:07:37.870050 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.870037 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wx97n\"" Sep 17 16:07:37.960565 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.960522 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b-proxy-tls\") pod \"switch-graph-48b6f-889dd88b7-x6cqt\" (UID: \"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b\") " pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:07:37.960565 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:37.960570 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b-openshift-service-ca-bundle\") pod \"switch-graph-48b6f-889dd88b7-x6cqt\" (UID: \"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b\") " pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:07:38.061440 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:38.061348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b-proxy-tls\") pod \"switch-graph-48b6f-889dd88b7-x6cqt\" (UID: \"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b\") " pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:07:38.061440 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:38.061401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b-openshift-service-ca-bundle\") pod \"switch-graph-48b6f-889dd88b7-x6cqt\" (UID: \"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b\") " pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:07:38.062004 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:38.061979 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b-openshift-service-ca-bundle\") pod \"switch-graph-48b6f-889dd88b7-x6cqt\" (UID: \"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b\") " pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:07:38.063820 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:38.063798 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b-proxy-tls\") pod \"switch-graph-48b6f-889dd88b7-x6cqt\" (UID: \"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b\") " pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:07:38.123367 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:38.123318 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:07:38.156323 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:38.156292 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:07:38.456899 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:38.456824 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt"] Sep 17 16:07:38.460297 ip-10-0-128-222 kubenswrapper[2580]: W0917 16:07:38.460263 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc43ce78_ee77_4449_b06c_d4ddb4da5e5b.slice/crio-7a01222ff81c28abe1cfce9d845f48edaf4b0a0443485de24ebef6d970bdf3df WatchSource:0}: Error finding container 7a01222ff81c28abe1cfce9d845f48edaf4b0a0443485de24ebef6d970bdf3df: Status 404 returned error can't find the container with id 7a01222ff81c28abe1cfce9d845f48edaf4b0a0443485de24ebef6d970bdf3df Sep 17 16:07:38.462210 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:38.462192 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 17 16:07:39.394847 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:39.394802 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" event={"ID":"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b","Type":"ContainerStarted","Data":"7a01222ff81c28abe1cfce9d845f48edaf4b0a0443485de24ebef6d970bdf3df"} Sep 17 16:07:41.405365 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:41.405272 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" event={"ID":"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b","Type":"ContainerStarted","Data":"bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872"} Sep 17 16:07:41.405816 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:41.405402 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:07:41.428671 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:41.428617 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" podStartSLOduration=1.824046562 podStartE2EDuration="4.428599816s" totalImagesPullingTime="2.604553254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:07:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:07:41.425908116 +0000 UTC m=+718.529153499" watchObservedRunningTime="2026-09-17 16:07:41.428599816 +0000 UTC m=+718.531845198" Sep 17 16:07:47.415012 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:47.414982 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:07:47.812556 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:47.812517 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt"] Sep 17 16:07:47.812763 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:47.812738 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" podUID="dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" containerName="switch-graph-48b6f" containerID="cri-o://bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872" gracePeriod=30 Sep 17 16:07:47.839848 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:47.839819 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:07:47.983535 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:47.983496 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:07:48.090103 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:48.090017 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:07:48.179563 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:48.179514 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:07:48.258573 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:48.258539 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:07:48.341544 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:48.341450 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:07:48.690051 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:48.689962 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:07:49.623505 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:49.623474 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:07:49.727651 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:49.727616 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:07:52.413659 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:52.413615 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" podUID="dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" containerName="switch-graph-48b6f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:07:57.413269 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:07:57.413210 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" podUID="dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" containerName="switch-graph-48b6f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:08:02.413036 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:02.412997 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" podUID="dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" containerName="switch-graph-48b6f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:08:02.413431 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:02.413100 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:08:07.413735 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:07.413695 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" podUID="dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" containerName="switch-graph-48b6f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:08:12.413157 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:12.413048 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" podUID="dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" containerName="switch-graph-48b6f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:08:17.412950 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:17.412907 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" podUID="dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" containerName="switch-graph-48b6f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:08:17.769715 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:17.769637 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:08:17.778900 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:17.778871 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p"] Sep 17 16:08:17.783776 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:17.783757 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:17.790751 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:17.790732 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\"" Sep 17 16:08:17.790861 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:08:17.790746 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"model-chainer-kube-rbac-proxy-sar-config\" is forbidden: User \"system:node:ip-10-0-128-222.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kserve-ci-e2e-test\": no relationship found between node 'ip-10-0-128-222.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" type="*v1.ConfigMap" Sep 17 16:08:17.791185 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:08:17.791162 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"model-chainer-7b8d5555b8-jkz7p\" is forbidden: User \"system:node:ip-10-0-128-222.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kserve-ci-e2e-test\": no relationship found between node 'ip-10-0-128-222.ec2.internal' and this object" podUID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:17.816075 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:17.816044 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p"] Sep 17 16:08:17.883657 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:17.883621 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-proxy-tls\") pod \"model-chainer-7b8d5555b8-jkz7p\" (UID: \"d0249e71-f1a4-4d5a-a487-2e8cc254cddd\") " pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:17.883788 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:17.883708 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-openshift-service-ca-bundle\") pod \"model-chainer-7b8d5555b8-jkz7p\" (UID: \"d0249e71-f1a4-4d5a-a487-2e8cc254cddd\") " pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:17.958957 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:17.958931 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:08:17.984286 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:17.984252 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-openshift-service-ca-bundle\") pod \"model-chainer-7b8d5555b8-jkz7p\" (UID: \"d0249e71-f1a4-4d5a-a487-2e8cc254cddd\") " pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:17.984703 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:17.984334 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-proxy-tls\") pod \"model-chainer-7b8d5555b8-jkz7p\" (UID: \"d0249e71-f1a4-4d5a-a487-2e8cc254cddd\") " pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:17.984703 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:08:17.984450 2580 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found Sep 17 16:08:17.984703 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:08:17.984523 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-proxy-tls podName:d0249e71-f1a4-4d5a-a487-2e8cc254cddd nodeName:}" failed. No retries permitted until 2026-09-17 16:08:18.484502152 +0000 UTC m=+755.587747512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-proxy-tls") pod "model-chainer-7b8d5555b8-jkz7p" (UID: "d0249e71-f1a4-4d5a-a487-2e8cc254cddd") : secret "model-chainer-serving-cert" not found Sep 17 16:08:17.984902 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:17.984884 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-openshift-service-ca-bundle\") pod \"model-chainer-7b8d5555b8-jkz7p\" (UID: \"d0249e71-f1a4-4d5a-a487-2e8cc254cddd\") " pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:18.085212 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.085170 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b-openshift-service-ca-bundle\") pod \"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b\" (UID: \"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b\") " Sep 17 16:08:18.085440 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.085256 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b-proxy-tls\") pod \"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b\" (UID: \"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b\") " Sep 17 16:08:18.085583 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.085560 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" (UID: "dc43ce78-ee77-4449-b06c-d4ddb4da5e5b"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 16:08:18.087407 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.087387 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" (UID: "dc43ce78-ee77-4449-b06c-d4ddb4da5e5b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 16:08:18.092069 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.092040 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:08:18.185952 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.185916 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b-openshift-service-ca-bundle\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:08:18.185952 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.185946 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b-proxy-tls\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:08:18.488609 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.488521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-proxy-tls\") pod \"model-chainer-7b8d5555b8-jkz7p\" (UID: \"d0249e71-f1a4-4d5a-a487-2e8cc254cddd\") " pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:18.490875 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.490855 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-proxy-tls\") pod \"model-chainer-7b8d5555b8-jkz7p\" (UID: \"d0249e71-f1a4-4d5a-a487-2e8cc254cddd\") " pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:18.555904 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.555867 2580 generic.go:356] "Generic (PLEG): container finished" podID="dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" containerID="bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872" exitCode=0 Sep 17 16:08:18.556061 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.555910 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" Sep 17 16:08:18.556061 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.555954 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" event={"ID":"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b","Type":"ContainerDied","Data":"bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872"} Sep 17 16:08:18.556061 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.556003 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt" event={"ID":"dc43ce78-ee77-4449-b06c-d4ddb4da5e5b","Type":"ContainerDied","Data":"7a01222ff81c28abe1cfce9d845f48edaf4b0a0443485de24ebef6d970bdf3df"} Sep 17 16:08:18.556061 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.556025 2580 scope.go:117] "RemoveContainer" containerID="bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872" Sep 17 16:08:18.564795 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.564777 2580 scope.go:117] "RemoveContainer" containerID="bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872" Sep 17 16:08:18.565096 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:08:18.565072 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872\": container with ID starting with bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872 not found: ID does not exist" containerID="bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872" Sep 17 16:08:18.565191 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.565102 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872"} err="failed to get container status \"bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872\": rpc error: code = NotFound desc = could not find container \"bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872\": container with ID starting with bb4c0ab19449f4c01c25e222ae591a1e95119547982c07c250aeff703e302872 not found: ID does not exist" Sep 17 16:08:18.587900 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.587873 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt"] Sep 17 16:08:18.595245 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.595205 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-48b6f-889dd88b7-x6cqt"] Sep 17 16:08:18.694178 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.694144 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:18.975817 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:18.975780 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p"] Sep 17 16:08:18.979003 ip-10-0-128-222 kubenswrapper[2580]: W0917 16:08:18.978976 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0249e71_f1a4_4d5a_a487_2e8cc254cddd.slice/crio-b15966c531b8169c9c747369934fa9362103d8d8a22cec6522d0934d653ed7b5 WatchSource:0}: Error finding container b15966c531b8169c9c747369934fa9362103d8d8a22cec6522d0934d653ed7b5: Status 404 returned error can't find the container with id b15966c531b8169c9c747369934fa9362103d8d8a22cec6522d0934d653ed7b5 Sep 17 16:08:19.373907 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:19.373873 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Sep 17 16:08:19.493257 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:19.493207 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" path="/var/lib/kubelet/pods/dc43ce78-ee77-4449-b06c-d4ddb4da5e5b/volumes" Sep 17 16:08:19.562270 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:19.562233 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" event={"ID":"d0249e71-f1a4-4d5a-a487-2e8cc254cddd","Type":"ContainerStarted","Data":"3cc445c3d4b1016fdd2c595d76f8156497cdd636e7a7e7a0c44891dea437568f"} Sep 17 16:08:19.562439 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:19.562293 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" event={"ID":"d0249e71-f1a4-4d5a-a487-2e8cc254cddd","Type":"ContainerStarted","Data":"b15966c531b8169c9c747369934fa9362103d8d8a22cec6522d0934d653ed7b5"} Sep 17 16:08:19.562439 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:19.562370 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:19.593575 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:19.593526 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" podStartSLOduration=2.59350961 podStartE2EDuration="2.59350961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:08:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:08:19.588961047 +0000 UTC m=+756.692206439" watchObservedRunningTime="2026-09-17 16:08:19.59350961 +0000 UTC m=+756.696754992" Sep 17 16:08:25.571260 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:25.571208 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:28.509049 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:28.509015 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p"] Sep 17 16:08:28.509546 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:28.509306 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" podUID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" containerName="model-chainer" containerID="cri-o://3cc445c3d4b1016fdd2c595d76f8156497cdd636e7a7e7a0c44891dea437568f" gracePeriod=30 Sep 17 16:08:28.565021 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:28.564988 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:08:28.725588 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:28.725559 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:08:28.829729 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:28.829699 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:08:28.934590 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:28.934540 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:08:29.040636 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:29.040602 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:08:29.170478 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:29.170394 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:08:29.267852 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:29.267821 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:08:29.401477 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:29.401446 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:08:30.570242 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:30.570178 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" podUID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:08:35.570505 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:35.570463 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" podUID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:08:40.570051 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:40.570000 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" podUID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:08:40.570452 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:40.570110 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:45.569591 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:45.569553 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" podUID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:08:48.178478 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.178435 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:08:48.210856 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.210815 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8"] Sep 17 16:08:48.211179 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.211163 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" containerName="switch-graph-48b6f" Sep 17 16:08:48.211299 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.211183 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" containerName="switch-graph-48b6f" Sep 17 16:08:48.211362 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.211297 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc43ce78-ee77-4449-b06c-d4ddb4da5e5b" containerName="switch-graph-48b6f" Sep 17 16:08:48.215124 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.215101 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:08:48.223760 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.223734 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-0dabd-serving-cert\"" Sep 17 16:08:48.223857 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.223806 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-0dabd-kube-rbac-proxy-sar-config\"" Sep 17 16:08:48.240189 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.240160 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8"] Sep 17 16:08:48.310794 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.310755 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5eef1539-7576-4adc-ad2d-5c4135476663-proxy-tls\") pod \"switch-graph-0dabd-b566dcb87-59wr8\" (UID: \"5eef1539-7576-4adc-ad2d-5c4135476663\") " pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:08:48.310987 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.310834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef1539-7576-4adc-ad2d-5c4135476663-openshift-service-ca-bundle\") pod \"switch-graph-0dabd-b566dcb87-59wr8\" (UID: \"5eef1539-7576-4adc-ad2d-5c4135476663\") " pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:08:48.386724 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.386685 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:08:48.411744 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.411707 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5eef1539-7576-4adc-ad2d-5c4135476663-proxy-tls\") pod \"switch-graph-0dabd-b566dcb87-59wr8\" (UID: \"5eef1539-7576-4adc-ad2d-5c4135476663\") " pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:08:48.411902 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.411760 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef1539-7576-4adc-ad2d-5c4135476663-openshift-service-ca-bundle\") pod \"switch-graph-0dabd-b566dcb87-59wr8\" (UID: \"5eef1539-7576-4adc-ad2d-5c4135476663\") " pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:08:48.412412 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.412392 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef1539-7576-4adc-ad2d-5c4135476663-openshift-service-ca-bundle\") pod \"switch-graph-0dabd-b566dcb87-59wr8\" (UID: \"5eef1539-7576-4adc-ad2d-5c4135476663\") " pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:08:48.414049 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.414033 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5eef1539-7576-4adc-ad2d-5c4135476663-proxy-tls\") pod \"switch-graph-0dabd-b566dcb87-59wr8\" (UID: \"5eef1539-7576-4adc-ad2d-5c4135476663\") " pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:08:48.524927 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.524843 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:08:48.802639 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:48.802614 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8"] Sep 17 16:08:48.803972 ip-10-0-128-222 kubenswrapper[2580]: W0917 16:08:48.803946 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eef1539_7576_4adc_ad2d_5c4135476663.slice/crio-2d31af2362e331fb8faa5a3f15b2ea845b7edaa1b5cda8e8ea5e110f26306feb WatchSource:0}: Error finding container 2d31af2362e331fb8faa5a3f15b2ea845b7edaa1b5cda8e8ea5e110f26306feb: Status 404 returned error can't find the container with id 2d31af2362e331fb8faa5a3f15b2ea845b7edaa1b5cda8e8ea5e110f26306feb Sep 17 16:08:49.678014 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:49.677976 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" event={"ID":"5eef1539-7576-4adc-ad2d-5c4135476663","Type":"ContainerStarted","Data":"2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40"} Sep 17 16:08:49.678413 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:49.678030 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" event={"ID":"5eef1539-7576-4adc-ad2d-5c4135476663","Type":"ContainerStarted","Data":"2d31af2362e331fb8faa5a3f15b2ea845b7edaa1b5cda8e8ea5e110f26306feb"} Sep 17 16:08:49.678413 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:49.678094 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:08:49.713699 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:49.713648 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" podStartSLOduration=1.7136304249999998 podStartE2EDuration="1.713630425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:08:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:08:49.712600577 +0000 UTC m=+786.815845958" watchObservedRunningTime="2026-09-17 16:08:49.713630425 +0000 UTC m=+786.816875809" Sep 17 16:08:50.570127 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:50.570087 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" podUID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:08:55.569972 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:55.569932 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" podUID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:08:55.688466 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:55.688433 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:08:58.713115 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:58.713068 2580 generic.go:356] "Generic (PLEG): container finished" podID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" containerID="3cc445c3d4b1016fdd2c595d76f8156497cdd636e7a7e7a0c44891dea437568f" exitCode=0 Sep 17 16:08:58.713672 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:58.713154 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" event={"ID":"d0249e71-f1a4-4d5a-a487-2e8cc254cddd","Type":"ContainerDied","Data":"3cc445c3d4b1016fdd2c595d76f8156497cdd636e7a7e7a0c44891dea437568f"} Sep 17 16:08:59.146887 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:59.146863 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:59.197405 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:59.197373 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-openshift-service-ca-bundle\") pod \"d0249e71-f1a4-4d5a-a487-2e8cc254cddd\" (UID: \"d0249e71-f1a4-4d5a-a487-2e8cc254cddd\") " Sep 17 16:08:59.197566 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:59.197442 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-proxy-tls\") pod \"d0249e71-f1a4-4d5a-a487-2e8cc254cddd\" (UID: \"d0249e71-f1a4-4d5a-a487-2e8cc254cddd\") " Sep 17 16:08:59.197788 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:59.197761 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "d0249e71-f1a4-4d5a-a487-2e8cc254cddd" (UID: "d0249e71-f1a4-4d5a-a487-2e8cc254cddd"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 16:08:59.199446 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:59.199408 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "d0249e71-f1a4-4d5a-a487-2e8cc254cddd" (UID: "d0249e71-f1a4-4d5a-a487-2e8cc254cddd"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 16:08:59.297997 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:59.297968 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-openshift-service-ca-bundle\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:08:59.297997 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:59.297997 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/d0249e71-f1a4-4d5a-a487-2e8cc254cddd-proxy-tls\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:08:59.718066 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:59.717976 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" event={"ID":"d0249e71-f1a4-4d5a-a487-2e8cc254cddd","Type":"ContainerDied","Data":"b15966c531b8169c9c747369934fa9362103d8d8a22cec6522d0934d653ed7b5"} Sep 17 16:08:59.718066 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:59.718039 2580 scope.go:117] "RemoveContainer" containerID="3cc445c3d4b1016fdd2c595d76f8156497cdd636e7a7e7a0c44891dea437568f" Sep 17 16:08:59.718066 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:59.717992 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p" Sep 17 16:08:59.754974 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:59.754943 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p"] Sep 17 16:08:59.762586 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:08:59.762561 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7b8d5555b8-jkz7p"] Sep 17 16:09:01.493129 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:01.493095 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" path="/var/lib/kubelet/pods/d0249e71-f1a4-4d5a-a487-2e8cc254cddd/volumes" Sep 17 16:09:28.760642 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:28.760607 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf"] Sep 17 16:09:28.761137 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:28.761004 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" containerName="model-chainer" Sep 17 16:09:28.761137 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:28.761022 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" containerName="model-chainer" Sep 17 16:09:28.761137 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:28.761117 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0249e71-f1a4-4d5a-a487-2e8cc254cddd" containerName="model-chainer" Sep 17 16:09:28.764043 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:28.764021 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:09:28.766663 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:28.766644 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-4f67a-serving-cert\"" Sep 17 16:09:28.767168 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:28.767152 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-4f67a-kube-rbac-proxy-sar-config\"" Sep 17 16:09:28.776616 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:28.776595 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf"] Sep 17 16:09:28.790190 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:28.790132 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:09:28.941914 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:28.941873 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df92046-3771-4262-9328-c69141335e45-openshift-service-ca-bundle\") pod \"sequence-graph-4f67a-5789f8f4f9-mpwtf\" (UID: \"8df92046-3771-4262-9328-c69141335e45\") " pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:09:28.942096 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:28.941933 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8df92046-3771-4262-9328-c69141335e45-proxy-tls\") pod \"sequence-graph-4f67a-5789f8f4f9-mpwtf\" (UID: \"8df92046-3771-4262-9328-c69141335e45\") " pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:09:29.042825 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:29.042799 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df92046-3771-4262-9328-c69141335e45-openshift-service-ca-bundle\") pod \"sequence-graph-4f67a-5789f8f4f9-mpwtf\" (UID: \"8df92046-3771-4262-9328-c69141335e45\") " pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:09:29.042973 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:29.042860 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8df92046-3771-4262-9328-c69141335e45-proxy-tls\") pod \"sequence-graph-4f67a-5789f8f4f9-mpwtf\" (UID: \"8df92046-3771-4262-9328-c69141335e45\") " pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:09:29.043463 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:29.043432 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df92046-3771-4262-9328-c69141335e45-openshift-service-ca-bundle\") pod \"sequence-graph-4f67a-5789f8f4f9-mpwtf\" (UID: \"8df92046-3771-4262-9328-c69141335e45\") " pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:09:29.045248 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:29.045229 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8df92046-3771-4262-9328-c69141335e45-proxy-tls\") pod \"sequence-graph-4f67a-5789f8f4f9-mpwtf\" (UID: \"8df92046-3771-4262-9328-c69141335e45\") " pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:09:29.073228 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:29.073185 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:09:29.347756 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:29.347680 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf"] Sep 17 16:09:29.356034 ip-10-0-128-222 kubenswrapper[2580]: W0917 16:09:29.356002 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8df92046_3771_4262_9328_c69141335e45.slice/crio-4efef6ea47a7a9a52cc99805d352ad14276ea536d455020fdd7817275ccdd5ca WatchSource:0}: Error finding container 4efef6ea47a7a9a52cc99805d352ad14276ea536d455020fdd7817275ccdd5ca: Status 404 returned error can't find the container with id 4efef6ea47a7a9a52cc99805d352ad14276ea536d455020fdd7817275ccdd5ca Sep 17 16:09:29.838000 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:29.837962 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" event={"ID":"8df92046-3771-4262-9328-c69141335e45","Type":"ContainerStarted","Data":"632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1"} Sep 17 16:09:29.838424 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:29.838019 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" event={"ID":"8df92046-3771-4262-9328-c69141335e45","Type":"ContainerStarted","Data":"4efef6ea47a7a9a52cc99805d352ad14276ea536d455020fdd7817275ccdd5ca"} Sep 17 16:09:29.838424 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:29.838077 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:09:29.859930 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:29.859880 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" podStartSLOduration=1.859861221 podStartE2EDuration="1.859861221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:09:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:09:29.859146832 +0000 UTC m=+826.962392213" watchObservedRunningTime="2026-09-17 16:09:29.859861221 +0000 UTC m=+826.963106584" Sep 17 16:09:30.003444 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:30.003401 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:09:35.846913 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:09:35.846884 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:10:43.305777 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:10:43.305747 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:10:43.306339 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:10:43.305969 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:10:43.313046 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:10:43.313026 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:10:43.313251 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:10:43.313233 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:10:43.315406 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:10:43.315386 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:10:43.315575 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:10:43.315557 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:11:41.806727 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:11:41.806692 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:13:38.064950 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:13:38.064866 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:13:38.144502 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:13:38.144470 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:13:51.433892 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:13:51.433859 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:15:43.329190 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:15:43.329158 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:15:43.330776 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:15:43.330757 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:15:43.336671 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:15:43.336651 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:15:43.337918 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:15:43.337901 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:15:43.339251 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:15:43.339236 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:15:43.340139 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:15:43.340122 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:16:38.016577 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:16:38.016493 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:02.786715 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:02.786679 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8"] Sep 17 16:17:02.787245 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:02.786879 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" podUID="5eef1539-7576-4adc-ad2d-5c4135476663" containerName="switch-graph-0dabd" containerID="cri-o://2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40" gracePeriod=30 Sep 17 16:17:02.814910 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:02.814879 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:02.963051 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:02.963019 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:03.056368 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:03.056335 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:03.135784 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:03.135745 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:03.211983 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:03.211950 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:03.426279 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:03.426172 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:03.521877 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:03.521845 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:05.686923 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:05.686877 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" podUID="5eef1539-7576-4adc-ad2d-5c4135476663" containerName="switch-graph-0dabd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:17:10.686761 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:10.686719 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" podUID="5eef1539-7576-4adc-ad2d-5c4135476663" containerName="switch-graph-0dabd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:17:15.686336 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:15.686300 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" podUID="5eef1539-7576-4adc-ad2d-5c4135476663" containerName="switch-graph-0dabd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:17:15.686717 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:15.686401 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:17:20.686270 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:20.686208 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" podUID="5eef1539-7576-4adc-ad2d-5c4135476663" containerName="switch-graph-0dabd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:17:25.686100 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:25.686056 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" podUID="5eef1539-7576-4adc-ad2d-5c4135476663" containerName="switch-graph-0dabd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:17:30.686592 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:30.686550 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" podUID="5eef1539-7576-4adc-ad2d-5c4135476663" containerName="switch-graph-0dabd" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:17:32.927160 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:32.927138 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:17:33.012022 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.011988 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5eef1539-7576-4adc-ad2d-5c4135476663-proxy-tls\") pod \"5eef1539-7576-4adc-ad2d-5c4135476663\" (UID: \"5eef1539-7576-4adc-ad2d-5c4135476663\") " Sep 17 16:17:33.012178 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.012072 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef1539-7576-4adc-ad2d-5c4135476663-openshift-service-ca-bundle\") pod \"5eef1539-7576-4adc-ad2d-5c4135476663\" (UID: \"5eef1539-7576-4adc-ad2d-5c4135476663\") " Sep 17 16:17:33.012461 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.012434 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eef1539-7576-4adc-ad2d-5c4135476663-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "5eef1539-7576-4adc-ad2d-5c4135476663" (UID: "5eef1539-7576-4adc-ad2d-5c4135476663"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 16:17:33.014113 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.014092 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eef1539-7576-4adc-ad2d-5c4135476663-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5eef1539-7576-4adc-ad2d-5c4135476663" (UID: "5eef1539-7576-4adc-ad2d-5c4135476663"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 16:17:33.112609 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.112582 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5eef1539-7576-4adc-ad2d-5c4135476663-proxy-tls\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:17:33.112728 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.112613 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5eef1539-7576-4adc-ad2d-5c4135476663-openshift-service-ca-bundle\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:17:33.677708 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.677675 2580 generic.go:356] "Generic (PLEG): container finished" podID="5eef1539-7576-4adc-ad2d-5c4135476663" containerID="2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40" exitCode=0 Sep 17 16:17:33.677970 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.677717 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" event={"ID":"5eef1539-7576-4adc-ad2d-5c4135476663","Type":"ContainerDied","Data":"2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40"} Sep 17 16:17:33.677970 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.677737 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" Sep 17 16:17:33.677970 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.677758 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8" event={"ID":"5eef1539-7576-4adc-ad2d-5c4135476663","Type":"ContainerDied","Data":"2d31af2362e331fb8faa5a3f15b2ea845b7edaa1b5cda8e8ea5e110f26306feb"} Sep 17 16:17:33.677970 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.677779 2580 scope.go:117] "RemoveContainer" containerID="2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40" Sep 17 16:17:33.688543 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.688520 2580 scope.go:117] "RemoveContainer" containerID="2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40" Sep 17 16:17:33.688861 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:17:33.688841 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40\": container with ID starting with 2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40 not found: ID does not exist" containerID="2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40" Sep 17 16:17:33.688934 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.688879 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40"} err="failed to get container status \"2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40\": rpc error: code = NotFound desc = could not find container \"2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40\": container with ID starting with 2d149897b10e3b4597a970a13465e05d525fa6f83bdeffa00f78995c99d8cf40 not found: ID does not exist" Sep 17 16:17:33.705780 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.705761 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8"] Sep 17 16:17:33.709748 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:33.709729 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-0dabd-b566dcb87-59wr8"] Sep 17 16:17:35.492666 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:35.492635 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5eef1539-7576-4adc-ad2d-5c4135476663" path="/var/lib/kubelet/pods/5eef1539-7576-4adc-ad2d-5c4135476663/volumes" Sep 17 16:17:43.245592 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:43.245536 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf"] Sep 17 16:17:43.246116 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:43.245756 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" podUID="8df92046-3771-4262-9328-c69141335e45" containerName="sequence-graph-4f67a" containerID="cri-o://632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1" gracePeriod=30 Sep 17 16:17:43.272411 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:43.272367 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:43.495461 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:43.495430 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:43.587423 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:43.587396 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:43.694273 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:43.694236 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:43.779842 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:43.779806 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:44.094788 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:44.094752 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:44.196839 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:44.196809 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:17:45.845229 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:45.845174 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" podUID="8df92046-3771-4262-9328-c69141335e45" containerName="sequence-graph-4f67a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:17:50.845961 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:50.845923 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" podUID="8df92046-3771-4262-9328-c69141335e45" containerName="sequence-graph-4f67a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:17:55.845378 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:55.845336 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" podUID="8df92046-3771-4262-9328-c69141335e45" containerName="sequence-graph-4f67a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:17:55.845769 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:17:55.845443 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:18:00.850199 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:00.850155 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" podUID="8df92046-3771-4262-9328-c69141335e45" containerName="sequence-graph-4f67a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:18:03.074923 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.074886 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h"] Sep 17 16:18:03.075355 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.075173 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eef1539-7576-4adc-ad2d-5c4135476663" containerName="switch-graph-0dabd" Sep 17 16:18:03.075355 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.075187 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eef1539-7576-4adc-ad2d-5c4135476663" containerName="switch-graph-0dabd" Sep 17 16:18:03.075355 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.075301 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eef1539-7576-4adc-ad2d-5c4135476663" containerName="switch-graph-0dabd" Sep 17 16:18:03.075938 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.075916 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:03.079444 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.079427 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:03.082468 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.082446 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-7c0d2-serving-cert\"" Sep 17 16:18:03.082722 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.082708 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-7c0d2-kube-rbac-proxy-sar-config\"" Sep 17 16:18:03.092953 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.092932 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h"] Sep 17 16:18:03.148560 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.148520 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28709baf-99e9-4f16-a098-0b4ad19d9f18-proxy-tls\") pod \"ensemble-graph-7c0d2-74876c585f-lhd4h\" (UID: \"28709baf-99e9-4f16-a098-0b4ad19d9f18\") " pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:03.148739 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.148622 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28709baf-99e9-4f16-a098-0b4ad19d9f18-openshift-service-ca-bundle\") pod \"ensemble-graph-7c0d2-74876c585f-lhd4h\" (UID: \"28709baf-99e9-4f16-a098-0b4ad19d9f18\") " pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:03.249026 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.248978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28709baf-99e9-4f16-a098-0b4ad19d9f18-proxy-tls\") pod \"ensemble-graph-7c0d2-74876c585f-lhd4h\" (UID: \"28709baf-99e9-4f16-a098-0b4ad19d9f18\") " pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:03.249267 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.249080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28709baf-99e9-4f16-a098-0b4ad19d9f18-openshift-service-ca-bundle\") pod \"ensemble-graph-7c0d2-74876c585f-lhd4h\" (UID: \"28709baf-99e9-4f16-a098-0b4ad19d9f18\") " pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:03.249267 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:18:03.249128 2580 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-7c0d2-serving-cert: secret "ensemble-graph-7c0d2-serving-cert" not found Sep 17 16:18:03.249267 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:18:03.249195 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28709baf-99e9-4f16-a098-0b4ad19d9f18-proxy-tls podName:28709baf-99e9-4f16-a098-0b4ad19d9f18 nodeName:}" failed. No retries permitted until 2026-09-17 16:18:03.749178723 +0000 UTC m=+1340.852424083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/28709baf-99e9-4f16-a098-0b4ad19d9f18-proxy-tls") pod "ensemble-graph-7c0d2-74876c585f-lhd4h" (UID: "28709baf-99e9-4f16-a098-0b4ad19d9f18") : secret "ensemble-graph-7c0d2-serving-cert" not found Sep 17 16:18:03.249781 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.249756 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28709baf-99e9-4f16-a098-0b4ad19d9f18-openshift-service-ca-bundle\") pod \"ensemble-graph-7c0d2-74876c585f-lhd4h\" (UID: \"28709baf-99e9-4f16-a098-0b4ad19d9f18\") " pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:03.318098 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.318062 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:03.753859 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.753818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28709baf-99e9-4f16-a098-0b4ad19d9f18-proxy-tls\") pod \"ensemble-graph-7c0d2-74876c585f-lhd4h\" (UID: \"28709baf-99e9-4f16-a098-0b4ad19d9f18\") " pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:03.756227 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.756193 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28709baf-99e9-4f16-a098-0b4ad19d9f18-proxy-tls\") pod \"ensemble-graph-7c0d2-74876c585f-lhd4h\" (UID: \"28709baf-99e9-4f16-a098-0b4ad19d9f18\") " pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:03.989739 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:03.989704 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:04.267869 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:04.267830 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h"] Sep 17 16:18:04.270690 ip-10-0-128-222 kubenswrapper[2580]: W0917 16:18:04.270662 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28709baf_99e9_4f16_a098_0b4ad19d9f18.slice/crio-587c2afa6747c51cf48bd5019d9a79ee3d86b62e94b6b8b2bbe53dc5cd548404 WatchSource:0}: Error finding container 587c2afa6747c51cf48bd5019d9a79ee3d86b62e94b6b8b2bbe53dc5cd548404: Status 404 returned error can't find the container with id 587c2afa6747c51cf48bd5019d9a79ee3d86b62e94b6b8b2bbe53dc5cd548404 Sep 17 16:18:04.797706 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:04.797665 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" event={"ID":"28709baf-99e9-4f16-a098-0b4ad19d9f18","Type":"ContainerStarted","Data":"09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820"} Sep 17 16:18:04.797994 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:04.797728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" event={"ID":"28709baf-99e9-4f16-a098-0b4ad19d9f18","Type":"ContainerStarted","Data":"587c2afa6747c51cf48bd5019d9a79ee3d86b62e94b6b8b2bbe53dc5cd548404"} Sep 17 16:18:04.797994 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:04.797778 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:04.822168 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:04.822112 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" podStartSLOduration=1.822097103 podStartE2EDuration="1.822097103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:18:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:18:04.822021548 +0000 UTC m=+1341.925266929" watchObservedRunningTime="2026-09-17 16:18:04.822097103 +0000 UTC m=+1341.925342484" Sep 17 16:18:05.846274 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:05.846175 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" podUID="8df92046-3771-4262-9328-c69141335e45" containerName="sequence-graph-4f67a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:18:10.807119 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:10.807080 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:10.846857 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:10.846816 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" podUID="8df92046-3771-4262-9328-c69141335e45" containerName="sequence-graph-4f67a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:18:13.193457 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.193422 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h"] Sep 17 16:18:13.193840 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.193597 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" podUID="28709baf-99e9-4f16-a098-0b4ad19d9f18" containerName="ensemble-graph-7c0d2" containerID="cri-o://09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820" gracePeriod=30 Sep 17 16:18:13.200462 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.200436 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:13.389491 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.389465 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:13.398750 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.398723 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:18:13.495385 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.494806 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:13.531076 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.531042 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df92046-3771-4262-9328-c69141335e45-openshift-service-ca-bundle\") pod \"8df92046-3771-4262-9328-c69141335e45\" (UID: \"8df92046-3771-4262-9328-c69141335e45\") " Sep 17 16:18:13.531285 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.531089 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8df92046-3771-4262-9328-c69141335e45-proxy-tls\") pod \"8df92046-3771-4262-9328-c69141335e45\" (UID: \"8df92046-3771-4262-9328-c69141335e45\") " Sep 17 16:18:13.531492 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.531464 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8df92046-3771-4262-9328-c69141335e45-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "8df92046-3771-4262-9328-c69141335e45" (UID: "8df92046-3771-4262-9328-c69141335e45"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 16:18:13.533115 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.533095 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8df92046-3771-4262-9328-c69141335e45-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8df92046-3771-4262-9328-c69141335e45" (UID: "8df92046-3771-4262-9328-c69141335e45"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 16:18:13.606889 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.606854 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:13.631820 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.631785 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8df92046-3771-4262-9328-c69141335e45-openshift-service-ca-bundle\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:18:13.631820 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.631813 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8df92046-3771-4262-9328-c69141335e45-proxy-tls\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:18:13.727291 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.727255 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:13.832093 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.832058 2580 generic.go:356] "Generic (PLEG): container finished" podID="8df92046-3771-4262-9328-c69141335e45" containerID="632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1" exitCode=0 Sep 17 16:18:13.832304 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.832115 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" Sep 17 16:18:13.832304 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.832115 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" event={"ID":"8df92046-3771-4262-9328-c69141335e45","Type":"ContainerDied","Data":"632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1"} Sep 17 16:18:13.832304 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.832246 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf" event={"ID":"8df92046-3771-4262-9328-c69141335e45","Type":"ContainerDied","Data":"4efef6ea47a7a9a52cc99805d352ad14276ea536d455020fdd7817275ccdd5ca"} Sep 17 16:18:13.832304 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.832272 2580 scope.go:117] "RemoveContainer" containerID="632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1" Sep 17 16:18:13.840696 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.840676 2580 scope.go:117] "RemoveContainer" containerID="632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1" Sep 17 16:18:13.841013 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:18:13.840994 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1\": container with ID starting with 632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1 not found: ID does not exist" containerID="632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1" Sep 17 16:18:13.841056 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.841022 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1"} err="failed to get container status \"632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1\": rpc error: code = NotFound desc = could not find container \"632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1\": container with ID starting with 632a57cd47c74020c4f36039c5d25666cc6ea8ad9c387392ea179e15495464b1 not found: ID does not exist" Sep 17 16:18:13.849533 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.849511 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:13.871697 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.871668 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf"] Sep 17 16:18:13.878669 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:13.878646 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-4f67a-5789f8f4f9-mpwtf"] Sep 17 16:18:14.058330 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:14.058299 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:15.492401 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:15.492361 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8df92046-3771-4262-9328-c69141335e45" path="/var/lib/kubelet/pods/8df92046-3771-4262-9328-c69141335e45/volumes" Sep 17 16:18:15.805202 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:15.805163 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" podUID="28709baf-99e9-4f16-a098-0b4ad19d9f18" containerName="ensemble-graph-7c0d2" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:18:20.805904 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:20.805861 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" podUID="28709baf-99e9-4f16-a098-0b4ad19d9f18" containerName="ensemble-graph-7c0d2" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:18:25.805288 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:25.805248 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" podUID="28709baf-99e9-4f16-a098-0b4ad19d9f18" containerName="ensemble-graph-7c0d2" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:18:25.805677 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:25.805359 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:30.805729 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:30.805688 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" podUID="28709baf-99e9-4f16-a098-0b4ad19d9f18" containerName="ensemble-graph-7c0d2" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:18:35.805398 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:35.805353 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" podUID="28709baf-99e9-4f16-a098-0b4ad19d9f18" containerName="ensemble-graph-7c0d2" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:18:40.805462 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:40.805425 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" podUID="28709baf-99e9-4f16-a098-0b4ad19d9f18" containerName="ensemble-graph-7c0d2" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:18:43.336454 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.336430 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:43.451537 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.451481 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28709baf-99e9-4f16-a098-0b4ad19d9f18-proxy-tls\") pod \"28709baf-99e9-4f16-a098-0b4ad19d9f18\" (UID: \"28709baf-99e9-4f16-a098-0b4ad19d9f18\") " Sep 17 16:18:43.451537 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.451532 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28709baf-99e9-4f16-a098-0b4ad19d9f18-openshift-service-ca-bundle\") pod \"28709baf-99e9-4f16-a098-0b4ad19d9f18\" (UID: \"28709baf-99e9-4f16-a098-0b4ad19d9f18\") " Sep 17 16:18:43.451902 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.451876 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28709baf-99e9-4f16-a098-0b4ad19d9f18-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "28709baf-99e9-4f16-a098-0b4ad19d9f18" (UID: "28709baf-99e9-4f16-a098-0b4ad19d9f18"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 16:18:43.453447 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.453427 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28709baf-99e9-4f16-a098-0b4ad19d9f18-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "28709baf-99e9-4f16-a098-0b4ad19d9f18" (UID: "28709baf-99e9-4f16-a098-0b4ad19d9f18"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 16:18:43.552635 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.552610 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28709baf-99e9-4f16-a098-0b4ad19d9f18-openshift-service-ca-bundle\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:18:43.552635 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.552632 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28709baf-99e9-4f16-a098-0b4ad19d9f18-proxy-tls\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:18:43.654610 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.654574 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl"] Sep 17 16:18:43.654890 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.654863 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28709baf-99e9-4f16-a098-0b4ad19d9f18" containerName="ensemble-graph-7c0d2" Sep 17 16:18:43.654890 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.654880 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="28709baf-99e9-4f16-a098-0b4ad19d9f18" containerName="ensemble-graph-7c0d2" Sep 17 16:18:43.654890 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.654894 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8df92046-3771-4262-9328-c69141335e45" containerName="sequence-graph-4f67a" Sep 17 16:18:43.655126 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.654902 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8df92046-3771-4262-9328-c69141335e45" containerName="sequence-graph-4f67a" Sep 17 16:18:43.655126 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.654977 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="28709baf-99e9-4f16-a098-0b4ad19d9f18" containerName="ensemble-graph-7c0d2" Sep 17 16:18:43.655126 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.654993 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8df92046-3771-4262-9328-c69141335e45" containerName="sequence-graph-4f67a" Sep 17 16:18:43.658658 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.658639 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:18:43.661946 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.661922 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-48574-kube-rbac-proxy-sar-config\"" Sep 17 16:18:43.662041 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.662003 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-48574-serving-cert\"" Sep 17 16:18:43.677112 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.677089 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl"] Sep 17 16:18:43.680570 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.680547 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:43.754680 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.754617 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdf47d46-b8d6-493c-8891-89e143ddbca1-openshift-service-ca-bundle\") pod \"sequence-graph-48574-bd8ccdb8c-g8gxl\" (UID: \"bdf47d46-b8d6-493c-8891-89e143ddbca1\") " pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:18:43.754680 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.754662 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bdf47d46-b8d6-493c-8891-89e143ddbca1-proxy-tls\") pod \"sequence-graph-48574-bd8ccdb8c-g8gxl\" (UID: \"bdf47d46-b8d6-493c-8891-89e143ddbca1\") " pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:18:43.855086 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.855059 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdf47d46-b8d6-493c-8891-89e143ddbca1-openshift-service-ca-bundle\") pod \"sequence-graph-48574-bd8ccdb8c-g8gxl\" (UID: \"bdf47d46-b8d6-493c-8891-89e143ddbca1\") " pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:18:43.855205 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.855108 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bdf47d46-b8d6-493c-8891-89e143ddbca1-proxy-tls\") pod \"sequence-graph-48574-bd8ccdb8c-g8gxl\" (UID: \"bdf47d46-b8d6-493c-8891-89e143ddbca1\") " pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:18:43.855688 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.855667 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdf47d46-b8d6-493c-8891-89e143ddbca1-openshift-service-ca-bundle\") pod \"sequence-graph-48574-bd8ccdb8c-g8gxl\" (UID: \"bdf47d46-b8d6-493c-8891-89e143ddbca1\") " pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:18:43.857393 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.857377 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bdf47d46-b8d6-493c-8891-89e143ddbca1-proxy-tls\") pod \"sequence-graph-48574-bd8ccdb8c-g8gxl\" (UID: \"bdf47d46-b8d6-493c-8891-89e143ddbca1\") " pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:18:43.926571 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.926542 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:43.947353 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.947324 2580 generic.go:356] "Generic (PLEG): container finished" podID="28709baf-99e9-4f16-a098-0b4ad19d9f18" containerID="09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820" exitCode=0 Sep 17 16:18:43.947487 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.947367 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" event={"ID":"28709baf-99e9-4f16-a098-0b4ad19d9f18","Type":"ContainerDied","Data":"09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820"} Sep 17 16:18:43.947487 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.947401 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" Sep 17 16:18:43.947487 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.947451 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h" event={"ID":"28709baf-99e9-4f16-a098-0b4ad19d9f18","Type":"ContainerDied","Data":"587c2afa6747c51cf48bd5019d9a79ee3d86b62e94b6b8b2bbe53dc5cd548404"} Sep 17 16:18:43.947487 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.947475 2580 scope.go:117] "RemoveContainer" containerID="09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820" Sep 17 16:18:43.955480 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.955453 2580 scope.go:117] "RemoveContainer" containerID="09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820" Sep 17 16:18:43.955829 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:18:43.955807 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820\": container with ID starting with 09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820 not found: ID does not exist" containerID="09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820" Sep 17 16:18:43.955911 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.955839 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820"} err="failed to get container status \"09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820\": rpc error: code = NotFound desc = could not find container \"09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820\": container with ID starting with 09698704ee41218216aaaa5107abc24fb2916e30a24aa13b7add6187c4733820 not found: ID does not exist" Sep 17 16:18:43.967863 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.967843 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:18:43.997815 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:43.997788 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h"] Sep 17 16:18:44.010233 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:44.010165 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-7c0d2-74876c585f-lhd4h"] Sep 17 16:18:44.264517 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:44.264452 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl"] Sep 17 16:18:44.267939 ip-10-0-128-222 kubenswrapper[2580]: W0917 16:18:44.267913 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdf47d46_b8d6_493c_8891_89e143ddbca1.slice/crio-8961db62c7727c43e7a01b391d6ec77a282a4fbd1cf2c6e6659b4c4f1de4237e WatchSource:0}: Error finding container 8961db62c7727c43e7a01b391d6ec77a282a4fbd1cf2c6e6659b4c4f1de4237e: Status 404 returned error can't find the container with id 8961db62c7727c43e7a01b391d6ec77a282a4fbd1cf2c6e6659b4c4f1de4237e Sep 17 16:18:44.953742 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:44.953706 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" event={"ID":"bdf47d46-b8d6-493c-8891-89e143ddbca1","Type":"ContainerStarted","Data":"6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1"} Sep 17 16:18:44.954134 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:44.953755 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" event={"ID":"bdf47d46-b8d6-493c-8891-89e143ddbca1","Type":"ContainerStarted","Data":"8961db62c7727c43e7a01b391d6ec77a282a4fbd1cf2c6e6659b4c4f1de4237e"} Sep 17 16:18:44.954134 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:44.953795 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:18:45.492702 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:45.492671 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28709baf-99e9-4f16-a098-0b4ad19d9f18" path="/var/lib/kubelet/pods/28709baf-99e9-4f16-a098-0b4ad19d9f18/volumes" Sep 17 16:18:50.962987 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:50.962952 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:18:50.997968 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:50.997922 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" podStartSLOduration=7.997906215 podStartE2EDuration="7.997906215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:18:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:18:44.982580627 +0000 UTC m=+1382.085826009" watchObservedRunningTime="2026-09-17 16:18:50.997906215 +0000 UTC m=+1388.101151597" Sep 17 16:18:53.657518 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:53.657479 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl"] Sep 17 16:18:53.657930 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:53.657668 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" podUID="bdf47d46-b8d6-493c-8891-89e143ddbca1" containerName="sequence-graph-48574" containerID="cri-o://6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1" gracePeriod=30 Sep 17 16:18:53.674523 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:53.674489 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:53.856411 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:53.856376 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:53.959807 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:53.959734 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:54.067871 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:54.067839 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:54.188519 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:54.188490 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:54.568455 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:54.568425 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:18:55.961556 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:18:55.961518 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" podUID="bdf47d46-b8d6-493c-8891-89e143ddbca1" containerName="sequence-graph-48574" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:19:00.962032 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:00.961994 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" podUID="bdf47d46-b8d6-493c-8891-89e143ddbca1" containerName="sequence-graph-48574" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:19:05.961947 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:05.961905 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" podUID="bdf47d46-b8d6-493c-8891-89e143ddbca1" containerName="sequence-graph-48574" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:19:05.962414 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:05.962037 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:19:10.962011 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:10.961974 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" podUID="bdf47d46-b8d6-493c-8891-89e143ddbca1" containerName="sequence-graph-48574" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:19:13.548435 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.548389 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64"] Sep 17 16:19:13.551613 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.551591 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:19:13.554136 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.554111 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-1e5cb-serving-cert\"" Sep 17 16:19:13.554392 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.554371 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-1e5cb-kube-rbac-proxy-sar-config\"" Sep 17 16:19:13.554513 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.554479 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:19:13.562756 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.562733 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64"] Sep 17 16:19:13.691083 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.691043 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/113d312f-924a-45ab-9e61-4ca9d60086bf-openshift-service-ca-bundle\") pod \"ensemble-graph-1e5cb-df94dbf5b-cmf64\" (UID: \"113d312f-924a-45ab-9e61-4ca9d60086bf\") " pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:19:13.691083 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.691084 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/113d312f-924a-45ab-9e61-4ca9d60086bf-proxy-tls\") pod \"ensemble-graph-1e5cb-df94dbf5b-cmf64\" (UID: \"113d312f-924a-45ab-9e61-4ca9d60086bf\") " pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:19:13.791536 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.791496 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/113d312f-924a-45ab-9e61-4ca9d60086bf-openshift-service-ca-bundle\") pod \"ensemble-graph-1e5cb-df94dbf5b-cmf64\" (UID: \"113d312f-924a-45ab-9e61-4ca9d60086bf\") " pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:19:13.791536 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.791539 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/113d312f-924a-45ab-9e61-4ca9d60086bf-proxy-tls\") pod \"ensemble-graph-1e5cb-df94dbf5b-cmf64\" (UID: \"113d312f-924a-45ab-9e61-4ca9d60086bf\") " pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:19:13.792204 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.792177 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/113d312f-924a-45ab-9e61-4ca9d60086bf-openshift-service-ca-bundle\") pod \"ensemble-graph-1e5cb-df94dbf5b-cmf64\" (UID: \"113d312f-924a-45ab-9e61-4ca9d60086bf\") " pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:19:13.794018 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.793990 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/113d312f-924a-45ab-9e61-4ca9d60086bf-proxy-tls\") pod \"ensemble-graph-1e5cb-df94dbf5b-cmf64\" (UID: \"113d312f-924a-45ab-9e61-4ca9d60086bf\") " pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:19:13.809716 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.809651 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:19:13.861933 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:13.861898 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:19:14.141388 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:14.141307 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64"] Sep 17 16:19:14.144552 ip-10-0-128-222 kubenswrapper[2580]: W0917 16:19:14.144523 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113d312f_924a_45ab_9e61_4ca9d60086bf.slice/crio-d9b8a3b7d38c492a7128aeaeb4cab424678e4dce72ba0b922c07abeb46ed8f03 WatchSource:0}: Error finding container d9b8a3b7d38c492a7128aeaeb4cab424678e4dce72ba0b922c07abeb46ed8f03: Status 404 returned error can't find the container with id d9b8a3b7d38c492a7128aeaeb4cab424678e4dce72ba0b922c07abeb46ed8f03 Sep 17 16:19:15.067012 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:15.066972 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" event={"ID":"113d312f-924a-45ab-9e61-4ca9d60086bf","Type":"ContainerStarted","Data":"4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9"} Sep 17 16:19:15.067431 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:15.067031 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" event={"ID":"113d312f-924a-45ab-9e61-4ca9d60086bf","Type":"ContainerStarted","Data":"d9b8a3b7d38c492a7128aeaeb4cab424678e4dce72ba0b922c07abeb46ed8f03"} Sep 17 16:19:15.067431 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:15.067059 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:19:15.095660 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:15.095608 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" podStartSLOduration=2.095592346 podStartE2EDuration="2.095592346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:19:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:19:15.094258128 +0000 UTC m=+1412.197503512" watchObservedRunningTime="2026-09-17 16:19:15.095592346 +0000 UTC m=+1412.198837728" Sep 17 16:19:15.961737 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:15.961698 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" podUID="bdf47d46-b8d6-493c-8891-89e143ddbca1" containerName="sequence-graph-48574" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:19:20.962116 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:20.962078 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" podUID="bdf47d46-b8d6-493c-8891-89e143ddbca1" containerName="sequence-graph-48574" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:19:21.076939 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:21.076910 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:19:23.801267 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:23.801240 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:19:23.975502 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:23.975414 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdf47d46-b8d6-493c-8891-89e143ddbca1-openshift-service-ca-bundle\") pod \"bdf47d46-b8d6-493c-8891-89e143ddbca1\" (UID: \"bdf47d46-b8d6-493c-8891-89e143ddbca1\") " Sep 17 16:19:23.975638 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:23.975520 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bdf47d46-b8d6-493c-8891-89e143ddbca1-proxy-tls\") pod \"bdf47d46-b8d6-493c-8891-89e143ddbca1\" (UID: \"bdf47d46-b8d6-493c-8891-89e143ddbca1\") " Sep 17 16:19:23.975806 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:23.975780 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdf47d46-b8d6-493c-8891-89e143ddbca1-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "bdf47d46-b8d6-493c-8891-89e143ddbca1" (UID: "bdf47d46-b8d6-493c-8891-89e143ddbca1"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 16:19:23.977637 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:23.977617 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdf47d46-b8d6-493c-8891-89e143ddbca1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bdf47d46-b8d6-493c-8891-89e143ddbca1" (UID: "bdf47d46-b8d6-493c-8891-89e143ddbca1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 16:19:24.076637 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:24.076600 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bdf47d46-b8d6-493c-8891-89e143ddbca1-proxy-tls\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:19:24.076637 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:24.076629 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bdf47d46-b8d6-493c-8891-89e143ddbca1-openshift-service-ca-bundle\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:19:24.103901 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:24.103869 2580 generic.go:356] "Generic (PLEG): container finished" podID="bdf47d46-b8d6-493c-8891-89e143ddbca1" containerID="6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1" exitCode=0 Sep 17 16:19:24.104035 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:24.103914 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" Sep 17 16:19:24.104035 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:24.103960 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" event={"ID":"bdf47d46-b8d6-493c-8891-89e143ddbca1","Type":"ContainerDied","Data":"6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1"} Sep 17 16:19:24.104035 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:24.104008 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl" event={"ID":"bdf47d46-b8d6-493c-8891-89e143ddbca1","Type":"ContainerDied","Data":"8961db62c7727c43e7a01b391d6ec77a282a4fbd1cf2c6e6659b4c4f1de4237e"} Sep 17 16:19:24.104035 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:24.104030 2580 scope.go:117] "RemoveContainer" containerID="6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1" Sep 17 16:19:24.113034 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:24.113017 2580 scope.go:117] "RemoveContainer" containerID="6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1" Sep 17 16:19:24.113333 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:19:24.113304 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1\": container with ID starting with 6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1 not found: ID does not exist" containerID="6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1" Sep 17 16:19:24.113418 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:24.113346 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1"} err="failed to get container status \"6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1\": rpc error: code = NotFound desc = could not find container \"6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1\": container with ID starting with 6f681aa8b419aeb71e12a55896c22cd7db41c60c493c9320f36213b7d4c8e4c1 not found: ID does not exist" Sep 17 16:19:24.137758 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:24.137732 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl"] Sep 17 16:19:24.141329 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:24.141305 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-48574-bd8ccdb8c-g8gxl"] Sep 17 16:19:25.493451 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:25.493415 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdf47d46-b8d6-493c-8891-89e143ddbca1" path="/var/lib/kubelet/pods/bdf47d46-b8d6-493c-8891-89e143ddbca1/volumes" Sep 17 16:19:54.039846 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.039813 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:19:54.071055 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.071024 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5"] Sep 17 16:19:54.071340 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.071327 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdf47d46-b8d6-493c-8891-89e143ddbca1" containerName="sequence-graph-48574" Sep 17 16:19:54.071388 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.071342 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdf47d46-b8d6-493c-8891-89e143ddbca1" containerName="sequence-graph-48574" Sep 17 16:19:54.071423 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.071396 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdf47d46-b8d6-493c-8891-89e143ddbca1" containerName="sequence-graph-48574" Sep 17 16:19:54.075268 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.075251 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:19:54.079078 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.079053 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-26f37-serving-cert\"" Sep 17 16:19:54.079875 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.079856 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-26f37-kube-rbac-proxy-sar-config\"" Sep 17 16:19:54.089104 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.089082 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5"] Sep 17 16:19:54.098094 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.098065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57386200-0320-4e87-bbde-b550f3fecd1e-openshift-service-ca-bundle\") pod \"sequence-graph-26f37-55f4c6587d-gplb5\" (UID: \"57386200-0320-4e87-bbde-b550f3fecd1e\") " pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:19:54.098187 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.098099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57386200-0320-4e87-bbde-b550f3fecd1e-proxy-tls\") pod \"sequence-graph-26f37-55f4c6587d-gplb5\" (UID: \"57386200-0320-4e87-bbde-b550f3fecd1e\") " pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:19:54.199102 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.199070 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57386200-0320-4e87-bbde-b550f3fecd1e-openshift-service-ca-bundle\") pod \"sequence-graph-26f37-55f4c6587d-gplb5\" (UID: \"57386200-0320-4e87-bbde-b550f3fecd1e\") " pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:19:54.199296 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.199109 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57386200-0320-4e87-bbde-b550f3fecd1e-proxy-tls\") pod \"sequence-graph-26f37-55f4c6587d-gplb5\" (UID: \"57386200-0320-4e87-bbde-b550f3fecd1e\") " pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:19:54.199296 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:19:54.199249 2580 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-26f37-serving-cert: secret "sequence-graph-26f37-serving-cert" not found Sep 17 16:19:54.199395 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:19:54.199307 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57386200-0320-4e87-bbde-b550f3fecd1e-proxy-tls podName:57386200-0320-4e87-bbde-b550f3fecd1e nodeName:}" failed. No retries permitted until 2026-09-17 16:19:54.69929059 +0000 UTC m=+1451.802535949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/57386200-0320-4e87-bbde-b550f3fecd1e-proxy-tls") pod "sequence-graph-26f37-55f4c6587d-gplb5" (UID: "57386200-0320-4e87-bbde-b550f3fecd1e") : secret "sequence-graph-26f37-serving-cert" not found Sep 17 16:19:54.199750 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.199730 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57386200-0320-4e87-bbde-b550f3fecd1e-openshift-service-ca-bundle\") pod \"sequence-graph-26f37-55f4c6587d-gplb5\" (UID: \"57386200-0320-4e87-bbde-b550f3fecd1e\") " pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:19:54.348554 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.348503 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:19:54.703342 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.703255 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57386200-0320-4e87-bbde-b550f3fecd1e-proxy-tls\") pod \"sequence-graph-26f37-55f4c6587d-gplb5\" (UID: \"57386200-0320-4e87-bbde-b550f3fecd1e\") " pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:19:54.705719 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.705693 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57386200-0320-4e87-bbde-b550f3fecd1e-proxy-tls\") pod \"sequence-graph-26f37-55f4c6587d-gplb5\" (UID: \"57386200-0320-4e87-bbde-b550f3fecd1e\") " pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:19:54.985418 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:54.985328 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:19:55.259924 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:55.259813 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5"] Sep 17 16:19:55.262567 ip-10-0-128-222 kubenswrapper[2580]: W0917 16:19:55.262536 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57386200_0320_4e87_bbde_b550f3fecd1e.slice/crio-2881980217a6a2f9b72547c3c18adb1792a6787049ff873e2c62afeb20c5b03a WatchSource:0}: Error finding container 2881980217a6a2f9b72547c3c18adb1792a6787049ff873e2c62afeb20c5b03a: Status 404 returned error can't find the container with id 2881980217a6a2f9b72547c3c18adb1792a6787049ff873e2c62afeb20c5b03a Sep 17 16:19:56.226099 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:56.226065 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" event={"ID":"57386200-0320-4e87-bbde-b550f3fecd1e","Type":"ContainerStarted","Data":"8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2"} Sep 17 16:19:56.226291 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:56.226116 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" event={"ID":"57386200-0320-4e87-bbde-b550f3fecd1e","Type":"ContainerStarted","Data":"2881980217a6a2f9b72547c3c18adb1792a6787049ff873e2c62afeb20c5b03a"} Sep 17 16:19:56.226291 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:56.226200 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:19:56.245508 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:19:56.245464 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" podStartSLOduration=2.24545204 podStartE2EDuration="2.24545204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:19:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:19:56.244514655 +0000 UTC m=+1453.347760042" watchObservedRunningTime="2026-09-17 16:19:56.24545204 +0000 UTC m=+1453.348697422" Sep 17 16:20:02.236154 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:20:02.236118 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:20:43.356322 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:20:43.356296 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:20:43.358979 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:20:43.358957 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:20:43.363875 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:20:43.363850 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:20:43.366624 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:20:43.366606 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:20:43.366624 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:20:43.366614 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:20:43.369125 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:20:43.369108 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:21:01.663527 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:21:01.663493 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:21:56.043957 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:21:56.043917 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:23:40.564434 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:23:40.564399 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:23:40.651011 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:23:40.650977 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:25:03.192829 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:25:03.192796 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:25:22.890784 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:25:22.890751 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:25:43.385947 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:25:43.385921 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:25:43.388085 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:25:43.388058 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:25:43.393457 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:25:43.393436 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:25:43.395501 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:25:43.395481 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:25:43.395898 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:25:43.395877 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:25:43.397859 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:25:43.397843 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:27:28.228266 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:28.228173 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64"] Sep 17 16:27:28.230632 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:28.228409 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" podUID="113d312f-924a-45ab-9e61-4ca9d60086bf" containerName="ensemble-graph-1e5cb" containerID="cri-o://4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9" gracePeriod=30 Sep 17 16:27:28.379515 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:28.379480 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:27:29.535494 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:29.535458 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:27:31.075472 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:31.075435 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" podUID="113d312f-924a-45ab-9e61-4ca9d60086bf" containerName="ensemble-graph-1e5cb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:27:33.510518 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:33.510484 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:27:36.075040 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:36.075005 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" podUID="113d312f-924a-45ab-9e61-4ca9d60086bf" containerName="ensemble-graph-1e5cb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:27:41.075297 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:41.075257 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" podUID="113d312f-924a-45ab-9e61-4ca9d60086bf" containerName="ensemble-graph-1e5cb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:27:41.075672 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:41.075390 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:27:46.075074 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:46.075034 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" podUID="113d312f-924a-45ab-9e61-4ca9d60086bf" containerName="ensemble-graph-1e5cb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:27:51.074700 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:51.074657 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" podUID="113d312f-924a-45ab-9e61-4ca9d60086bf" containerName="ensemble-graph-1e5cb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:27:56.074836 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:56.074792 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" podUID="113d312f-924a-45ab-9e61-4ca9d60086bf" containerName="ensemble-graph-1e5cb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:27:58.362373 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:58.362349 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:27:58.385290 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:58.385253 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/113d312f-924a-45ab-9e61-4ca9d60086bf-openshift-service-ca-bundle\") pod \"113d312f-924a-45ab-9e61-4ca9d60086bf\" (UID: \"113d312f-924a-45ab-9e61-4ca9d60086bf\") " Sep 17 16:27:58.385492 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:58.385467 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/113d312f-924a-45ab-9e61-4ca9d60086bf-proxy-tls\") pod \"113d312f-924a-45ab-9e61-4ca9d60086bf\" (UID: \"113d312f-924a-45ab-9e61-4ca9d60086bf\") " Sep 17 16:27:58.385717 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:58.385689 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/113d312f-924a-45ab-9e61-4ca9d60086bf-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "113d312f-924a-45ab-9e61-4ca9d60086bf" (UID: "113d312f-924a-45ab-9e61-4ca9d60086bf"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 16:27:58.387876 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:58.387853 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/113d312f-924a-45ab-9e61-4ca9d60086bf-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "113d312f-924a-45ab-9e61-4ca9d60086bf" (UID: "113d312f-924a-45ab-9e61-4ca9d60086bf"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 16:27:58.486518 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:58.486461 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/113d312f-924a-45ab-9e61-4ca9d60086bf-openshift-service-ca-bundle\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:27:58.486518 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:58.486484 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/113d312f-924a-45ab-9e61-4ca9d60086bf-proxy-tls\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:27:59.055321 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:59.055287 2580 generic.go:356] "Generic (PLEG): container finished" podID="113d312f-924a-45ab-9e61-4ca9d60086bf" containerID="4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9" exitCode=0 Sep 17 16:27:59.055482 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:59.055327 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" Sep 17 16:27:59.055482 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:59.055360 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" event={"ID":"113d312f-924a-45ab-9e61-4ca9d60086bf","Type":"ContainerDied","Data":"4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9"} Sep 17 16:27:59.055482 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:59.055410 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64" event={"ID":"113d312f-924a-45ab-9e61-4ca9d60086bf","Type":"ContainerDied","Data":"d9b8a3b7d38c492a7128aeaeb4cab424678e4dce72ba0b922c07abeb46ed8f03"} Sep 17 16:27:59.055482 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:59.055433 2580 scope.go:117] "RemoveContainer" containerID="4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9" Sep 17 16:27:59.065444 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:59.065426 2580 scope.go:117] "RemoveContainer" containerID="4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9" Sep 17 16:27:59.065750 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:27:59.065719 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9\": container with ID starting with 4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9 not found: ID does not exist" containerID="4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9" Sep 17 16:27:59.065845 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:59.065758 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9"} err="failed to get container status \"4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9\": rpc error: code = NotFound desc = could not find container \"4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9\": container with ID starting with 4a9c226df8e2c5634f931ac659ef18bbe3da4d094301303ba88485c45cabacb9 not found: ID does not exist" Sep 17 16:27:59.091014 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:59.090994 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64"] Sep 17 16:27:59.097827 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:59.097806 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1e5cb-df94dbf5b-cmf64"] Sep 17 16:27:59.493157 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:27:59.493075 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="113d312f-924a-45ab-9e61-4ca9d60086bf" path="/var/lib/kubelet/pods/113d312f-924a-45ab-9e61-4ca9d60086bf/volumes" Sep 17 16:28:08.564202 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:08.564166 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5"] Sep 17 16:28:08.564649 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:08.564419 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" podUID="57386200-0320-4e87-bbde-b550f3fecd1e" containerName="sequence-graph-26f37" containerID="cri-o://8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2" gracePeriod=30 Sep 17 16:28:08.589003 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:08.588974 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:08.705523 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:08.705486 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:08.806963 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:08.806933 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:08.898804 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:08.898722 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:08.998577 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:08.998541 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:09.142165 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:09.142134 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:12.234272 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:12.234234 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" podUID="57386200-0320-4e87-bbde-b550f3fecd1e" containerName="sequence-graph-26f37" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:28:17.234001 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:17.233957 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" podUID="57386200-0320-4e87-bbde-b550f3fecd1e" containerName="sequence-graph-26f37" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:28:22.233847 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:22.233808 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" podUID="57386200-0320-4e87-bbde-b550f3fecd1e" containerName="sequence-graph-26f37" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:28:22.234264 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:22.233915 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:28:27.234137 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:27.234102 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" podUID="57386200-0320-4e87-bbde-b550f3fecd1e" containerName="sequence-graph-26f37" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:28:28.746736 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.746701 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9"] Sep 17 16:28:28.747114 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.747002 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113d312f-924a-45ab-9e61-4ca9d60086bf" containerName="ensemble-graph-1e5cb" Sep 17 16:28:28.747114 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.747012 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="113d312f-924a-45ab-9e61-4ca9d60086bf" containerName="ensemble-graph-1e5cb" Sep 17 16:28:28.747114 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.747079 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="113d312f-924a-45ab-9e61-4ca9d60086bf" containerName="ensemble-graph-1e5cb" Sep 17 16:28:28.749816 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.749794 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:28:28.752663 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.752635 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-7ce9a-serving-cert\"" Sep 17 16:28:28.752814 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.752800 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-7ce9a-kube-rbac-proxy-sar-config\"" Sep 17 16:28:28.766464 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.766438 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9"] Sep 17 16:28:28.766637 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.766514 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:28.805670 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.805634 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92c33261-3945-41ca-9bc1-ed2c56ebc8f0-openshift-service-ca-bundle\") pod \"splitter-graph-7ce9a-569dbd59-5z7h9\" (UID: \"92c33261-3945-41ca-9bc1-ed2c56ebc8f0\") " pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:28:28.805670 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.805676 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92c33261-3945-41ca-9bc1-ed2c56ebc8f0-proxy-tls\") pod \"splitter-graph-7ce9a-569dbd59-5z7h9\" (UID: \"92c33261-3945-41ca-9bc1-ed2c56ebc8f0\") " pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:28:28.886140 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.886109 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:28.906306 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.906277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92c33261-3945-41ca-9bc1-ed2c56ebc8f0-openshift-service-ca-bundle\") pod \"splitter-graph-7ce9a-569dbd59-5z7h9\" (UID: \"92c33261-3945-41ca-9bc1-ed2c56ebc8f0\") " pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:28:28.906445 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.906314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92c33261-3945-41ca-9bc1-ed2c56ebc8f0-proxy-tls\") pod \"splitter-graph-7ce9a-569dbd59-5z7h9\" (UID: \"92c33261-3945-41ca-9bc1-ed2c56ebc8f0\") " pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:28:28.906897 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.906873 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92c33261-3945-41ca-9bc1-ed2c56ebc8f0-openshift-service-ca-bundle\") pod \"splitter-graph-7ce9a-569dbd59-5z7h9\" (UID: \"92c33261-3945-41ca-9bc1-ed2c56ebc8f0\") " pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:28:28.908653 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:28.908633 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92c33261-3945-41ca-9bc1-ed2c56ebc8f0-proxy-tls\") pod \"splitter-graph-7ce9a-569dbd59-5z7h9\" (UID: \"92c33261-3945-41ca-9bc1-ed2c56ebc8f0\") " pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:28:29.065973 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:29.065942 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:28:29.337883 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:29.337855 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9"] Sep 17 16:28:29.339814 ip-10-0-128-222 kubenswrapper[2580]: W0917 16:28:29.339790 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c33261_3945_41ca_9bc1_ed2c56ebc8f0.slice/crio-a751e42ce5122557fc4b0819c260873e5e8f64df95d00d02995b01e3ebc5e61d WatchSource:0}: Error finding container a751e42ce5122557fc4b0819c260873e5e8f64df95d00d02995b01e3ebc5e61d: Status 404 returned error can't find the container with id a751e42ce5122557fc4b0819c260873e5e8f64df95d00d02995b01e3ebc5e61d Sep 17 16:28:30.177416 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:30.177380 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" event={"ID":"92c33261-3945-41ca-9bc1-ed2c56ebc8f0","Type":"ContainerStarted","Data":"c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d"} Sep 17 16:28:30.177804 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:30.177510 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:28:30.177804 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:30.177546 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" event={"ID":"92c33261-3945-41ca-9bc1-ed2c56ebc8f0","Type":"ContainerStarted","Data":"a751e42ce5122557fc4b0819c260873e5e8f64df95d00d02995b01e3ebc5e61d"} Sep 17 16:28:30.198905 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:30.198857 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" podStartSLOduration=2.198843306 podStartE2EDuration="2.198843306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:28:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:28:30.197032069 +0000 UTC m=+1967.300277452" watchObservedRunningTime="2026-09-17 16:28:30.198843306 +0000 UTC m=+1967.302088688" Sep 17 16:28:32.233980 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:32.233937 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" podUID="57386200-0320-4e87-bbde-b550f3fecd1e" containerName="sequence-graph-26f37" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:28:36.187886 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:36.187814 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:28:37.234721 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:37.234672 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" podUID="57386200-0320-4e87-bbde-b550f3fecd1e" containerName="sequence-graph-26f37" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:28:38.710582 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:38.710556 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:28:38.788236 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:38.788198 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57386200-0320-4e87-bbde-b550f3fecd1e-proxy-tls\") pod \"57386200-0320-4e87-bbde-b550f3fecd1e\" (UID: \"57386200-0320-4e87-bbde-b550f3fecd1e\") " Sep 17 16:28:38.788427 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:38.788406 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57386200-0320-4e87-bbde-b550f3fecd1e-openshift-service-ca-bundle\") pod \"57386200-0320-4e87-bbde-b550f3fecd1e\" (UID: \"57386200-0320-4e87-bbde-b550f3fecd1e\") " Sep 17 16:28:38.788840 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:38.788812 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57386200-0320-4e87-bbde-b550f3fecd1e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "57386200-0320-4e87-bbde-b550f3fecd1e" (UID: "57386200-0320-4e87-bbde-b550f3fecd1e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 16:28:38.789562 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:38.789542 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9"] Sep 17 16:28:38.789777 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:38.789743 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" podUID="92c33261-3945-41ca-9bc1-ed2c56ebc8f0" containerName="splitter-graph-7ce9a" containerID="cri-o://c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d" gracePeriod=30 Sep 17 16:28:38.790361 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:38.790340 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57386200-0320-4e87-bbde-b550f3fecd1e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "57386200-0320-4e87-bbde-b550f3fecd1e" (UID: "57386200-0320-4e87-bbde-b550f3fecd1e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 16:28:38.821616 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:38.821555 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:38.889104 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:38.889071 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/57386200-0320-4e87-bbde-b550f3fecd1e-openshift-service-ca-bundle\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:28:38.889104 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:38.889106 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/57386200-0320-4e87-bbde-b550f3fecd1e-proxy-tls\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:28:38.931358 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:38.931328 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:39.045690 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.045657 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:39.143789 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.143703 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:39.213876 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.213840 2580 generic.go:356] "Generic (PLEG): container finished" podID="57386200-0320-4e87-bbde-b550f3fecd1e" containerID="8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2" exitCode=0 Sep 17 16:28:39.214067 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.213887 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" Sep 17 16:28:39.214067 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.213897 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" event={"ID":"57386200-0320-4e87-bbde-b550f3fecd1e","Type":"ContainerDied","Data":"8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2"} Sep 17 16:28:39.214067 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.213949 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5" event={"ID":"57386200-0320-4e87-bbde-b550f3fecd1e","Type":"ContainerDied","Data":"2881980217a6a2f9b72547c3c18adb1792a6787049ff873e2c62afeb20c5b03a"} Sep 17 16:28:39.214067 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.213973 2580 scope.go:117] "RemoveContainer" containerID="8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2" Sep 17 16:28:39.223485 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.223465 2580 scope.go:117] "RemoveContainer" containerID="8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2" Sep 17 16:28:39.223801 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:28:39.223783 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2\": container with ID starting with 8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2 not found: ID does not exist" containerID="8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2" Sep 17 16:28:39.224023 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.223811 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2"} err="failed to get container status \"8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2\": rpc error: code = NotFound desc = could not find container \"8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2\": container with ID starting with 8b7c2e2c4b6abd5fc09aa605a53cffcc111c10d1f461be4c999b7d443e1314e2 not found: ID does not exist" Sep 17 16:28:39.240960 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.240927 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5"] Sep 17 16:28:39.249181 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.249147 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-26f37-55f4c6587d-gplb5"] Sep 17 16:28:39.253262 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.253237 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:39.473626 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.473522 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:39.492666 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.492635 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57386200-0320-4e87-bbde-b550f3fecd1e" path="/var/lib/kubelet/pods/57386200-0320-4e87-bbde-b550f3fecd1e/volumes" Sep 17 16:28:39.643354 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:39.643315 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:28:41.185335 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:41.185290 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" podUID="92c33261-3945-41ca-9bc1-ed2c56ebc8f0" containerName="splitter-graph-7ce9a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:28:46.185729 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:46.185682 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" podUID="92c33261-3945-41ca-9bc1-ed2c56ebc8f0" containerName="splitter-graph-7ce9a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:28:51.185485 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:51.185438 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" podUID="92c33261-3945-41ca-9bc1-ed2c56ebc8f0" containerName="splitter-graph-7ce9a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:28:51.185953 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:51.185545 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:28:56.185291 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:28:56.185241 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" podUID="92c33261-3945-41ca-9bc1-ed2c56ebc8f0" containerName="splitter-graph-7ce9a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:29:01.185981 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:01.185937 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" podUID="92c33261-3945-41ca-9bc1-ed2c56ebc8f0" containerName="splitter-graph-7ce9a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:29:06.185987 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:06.185946 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" podUID="92c33261-3945-41ca-9bc1-ed2c56ebc8f0" containerName="splitter-graph-7ce9a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:29:08.864741 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:08.864706 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:29:08.865265 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:08.865108 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25"] Sep 17 16:29:08.865420 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:08.865405 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57386200-0320-4e87-bbde-b550f3fecd1e" containerName="sequence-graph-26f37" Sep 17 16:29:08.865479 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:08.865421 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="57386200-0320-4e87-bbde-b550f3fecd1e" containerName="sequence-graph-26f37" Sep 17 16:29:08.865537 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:08.865485 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="57386200-0320-4e87-bbde-b550f3fecd1e" containerName="sequence-graph-26f37" Sep 17 16:29:08.868319 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:08.868297 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:29:08.872904 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:08.872883 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-37c70-serving-cert\"" Sep 17 16:29:08.872904 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:08.872883 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-37c70-kube-rbac-proxy-sar-config\"" Sep 17 16:29:08.881625 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:08.881598 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25"] Sep 17 16:29:08.941781 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:08.941747 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:29:09.017577 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.017542 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:29:09.030604 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.030578 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28ef2274-6b65-469d-9491-19ccb33a1564-proxy-tls\") pod \"switch-graph-37c70-86c98d65b-tnx25\" (UID: \"28ef2274-6b65-469d-9491-19ccb33a1564\") " pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:29:09.030719 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.030630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ef2274-6b65-469d-9491-19ccb33a1564-openshift-service-ca-bundle\") pod \"switch-graph-37c70-86c98d65b-tnx25\" (UID: \"28ef2274-6b65-469d-9491-19ccb33a1564\") " pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:29:09.131358 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.131269 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92c33261-3945-41ca-9bc1-ed2c56ebc8f0-proxy-tls\") pod \"92c33261-3945-41ca-9bc1-ed2c56ebc8f0\" (UID: \"92c33261-3945-41ca-9bc1-ed2c56ebc8f0\") " Sep 17 16:29:09.131358 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.131310 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92c33261-3945-41ca-9bc1-ed2c56ebc8f0-openshift-service-ca-bundle\") pod \"92c33261-3945-41ca-9bc1-ed2c56ebc8f0\" (UID: \"92c33261-3945-41ca-9bc1-ed2c56ebc8f0\") " Sep 17 16:29:09.131550 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.131380 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ef2274-6b65-469d-9491-19ccb33a1564-openshift-service-ca-bundle\") pod \"switch-graph-37c70-86c98d65b-tnx25\" (UID: \"28ef2274-6b65-469d-9491-19ccb33a1564\") " pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:29:09.131550 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.131529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28ef2274-6b65-469d-9491-19ccb33a1564-proxy-tls\") pod \"switch-graph-37c70-86c98d65b-tnx25\" (UID: \"28ef2274-6b65-469d-9491-19ccb33a1564\") " pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:29:09.131752 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.131729 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92c33261-3945-41ca-9bc1-ed2c56ebc8f0-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "92c33261-3945-41ca-9bc1-ed2c56ebc8f0" (UID: "92c33261-3945-41ca-9bc1-ed2c56ebc8f0"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 16:29:09.137333 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.132563 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ef2274-6b65-469d-9491-19ccb33a1564-openshift-service-ca-bundle\") pod \"switch-graph-37c70-86c98d65b-tnx25\" (UID: \"28ef2274-6b65-469d-9491-19ccb33a1564\") " pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:29:09.138458 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.138422 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92c33261-3945-41ca-9bc1-ed2c56ebc8f0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "92c33261-3945-41ca-9bc1-ed2c56ebc8f0" (UID: "92c33261-3945-41ca-9bc1-ed2c56ebc8f0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 16:29:09.139074 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.139049 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28ef2274-6b65-469d-9491-19ccb33a1564-proxy-tls\") pod \"switch-graph-37c70-86c98d65b-tnx25\" (UID: \"28ef2274-6b65-469d-9491-19ccb33a1564\") " pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:29:09.180180 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.180131 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:29:09.233003 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.232970 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/92c33261-3945-41ca-9bc1-ed2c56ebc8f0-proxy-tls\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:29:09.233116 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.233006 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/92c33261-3945-41ca-9bc1-ed2c56ebc8f0-openshift-service-ca-bundle\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:29:09.336361 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.336328 2580 generic.go:356] "Generic (PLEG): container finished" podID="92c33261-3945-41ca-9bc1-ed2c56ebc8f0" containerID="c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d" exitCode=0 Sep 17 16:29:09.336550 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.336386 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" Sep 17 16:29:09.336550 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.336419 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" event={"ID":"92c33261-3945-41ca-9bc1-ed2c56ebc8f0","Type":"ContainerDied","Data":"c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d"} Sep 17 16:29:09.336550 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.336500 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9" event={"ID":"92c33261-3945-41ca-9bc1-ed2c56ebc8f0","Type":"ContainerDied","Data":"a751e42ce5122557fc4b0819c260873e5e8f64df95d00d02995b01e3ebc5e61d"} Sep 17 16:29:09.336550 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.336524 2580 scope.go:117] "RemoveContainer" containerID="c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d" Sep 17 16:29:09.345030 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.345009 2580 scope.go:117] "RemoveContainer" containerID="c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d" Sep 17 16:29:09.345394 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:29:09.345371 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d\": container with ID starting with c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d not found: ID does not exist" containerID="c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d" Sep 17 16:29:09.345454 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.345404 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d"} err="failed to get container status \"c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d\": rpc error: code = NotFound desc = could not find container \"c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d\": container with ID starting with c17d11b666ba8cab57b62fa244501c209dd8bcc5ba07ab885d6531e7d527260d not found: ID does not exist" Sep 17 16:29:09.368102 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.368073 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9"] Sep 17 16:29:09.371350 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.371324 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-7ce9a-569dbd59-5z7h9"] Sep 17 16:29:09.462007 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.461914 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25"] Sep 17 16:29:09.466607 ip-10-0-128-222 kubenswrapper[2580]: W0917 16:29:09.466579 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28ef2274_6b65_469d_9491_19ccb33a1564.slice/crio-536c011a488ebbad7f81d12076a0ba0cfc8c73200cd576e9246d22d99dd9361f WatchSource:0}: Error finding container 536c011a488ebbad7f81d12076a0ba0cfc8c73200cd576e9246d22d99dd9361f: Status 404 returned error can't find the container with id 536c011a488ebbad7f81d12076a0ba0cfc8c73200cd576e9246d22d99dd9361f Sep 17 16:29:09.492917 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:09.492886 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92c33261-3945-41ca-9bc1-ed2c56ebc8f0" path="/var/lib/kubelet/pods/92c33261-3945-41ca-9bc1-ed2c56ebc8f0/volumes" Sep 17 16:29:10.343398 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:10.343359 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" event={"ID":"28ef2274-6b65-469d-9491-19ccb33a1564","Type":"ContainerStarted","Data":"b4220e051911a959dcaaba80761f691d667a61a1ea5412b3523fafbd334b3986"} Sep 17 16:29:10.343776 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:10.343414 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" event={"ID":"28ef2274-6b65-469d-9491-19ccb33a1564","Type":"ContainerStarted","Data":"536c011a488ebbad7f81d12076a0ba0cfc8c73200cd576e9246d22d99dd9361f"} Sep 17 16:29:10.343776 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:10.343446 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:29:10.368439 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:10.368390 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" podStartSLOduration=2.368369622 podStartE2EDuration="2.368369622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:29:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:29:10.365343779 +0000 UTC m=+2007.468589161" watchObservedRunningTime="2026-09-17 16:29:10.368369622 +0000 UTC m=+2007.471615003" Sep 17 16:29:16.353328 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:16.353293 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:29:39.143979 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.143939 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:29:39.146499 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.146474 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr"] Sep 17 16:29:39.146826 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.146810 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92c33261-3945-41ca-9bc1-ed2c56ebc8f0" containerName="splitter-graph-7ce9a" Sep 17 16:29:39.146899 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.146830 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="92c33261-3945-41ca-9bc1-ed2c56ebc8f0" containerName="splitter-graph-7ce9a" Sep 17 16:29:39.146954 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.146929 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="92c33261-3945-41ca-9bc1-ed2c56ebc8f0" containerName="splitter-graph-7ce9a" Sep 17 16:29:39.149594 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.149576 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:29:39.152208 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.152188 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-6d5c6-kube-rbac-proxy-sar-config\"" Sep 17 16:29:39.152312 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.152190 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-6d5c6-serving-cert\"" Sep 17 16:29:39.170824 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.170805 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr"] Sep 17 16:29:39.237583 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.237548 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1d3edbeb-6c3d-4098-ac79-3820ea076531-proxy-tls\") pod \"splitter-graph-6d5c6-5d4986d579-ngwcr\" (UID: \"1d3edbeb-6c3d-4098-ac79-3820ea076531\") " pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:29:39.237583 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.237616 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d3edbeb-6c3d-4098-ac79-3820ea076531-openshift-service-ca-bundle\") pod \"splitter-graph-6d5c6-5d4986d579-ngwcr\" (UID: \"1d3edbeb-6c3d-4098-ac79-3820ea076531\") " pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:29:39.238945 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.238907 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:29:39.338828 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.338792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1d3edbeb-6c3d-4098-ac79-3820ea076531-proxy-tls\") pod \"splitter-graph-6d5c6-5d4986d579-ngwcr\" (UID: \"1d3edbeb-6c3d-4098-ac79-3820ea076531\") " pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:29:39.339018 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.338856 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d3edbeb-6c3d-4098-ac79-3820ea076531-openshift-service-ca-bundle\") pod \"splitter-graph-6d5c6-5d4986d579-ngwcr\" (UID: \"1d3edbeb-6c3d-4098-ac79-3820ea076531\") " pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:29:39.339526 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.339496 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d3edbeb-6c3d-4098-ac79-3820ea076531-openshift-service-ca-bundle\") pod \"splitter-graph-6d5c6-5d4986d579-ngwcr\" (UID: \"1d3edbeb-6c3d-4098-ac79-3820ea076531\") " pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:29:39.341240 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.341202 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1d3edbeb-6c3d-4098-ac79-3820ea076531-proxy-tls\") pod \"splitter-graph-6d5c6-5d4986d579-ngwcr\" (UID: \"1d3edbeb-6c3d-4098-ac79-3820ea076531\") " pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:29:39.460086 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.460008 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:29:39.739985 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:39.739399 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr"] Sep 17 16:29:39.745021 ip-10-0-128-222 kubenswrapper[2580]: W0917 16:29:39.744988 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3edbeb_6c3d_4098_ac79_3820ea076531.slice/crio-5c23ac0445ebaff5300340da6613a5c0d2788876c0da44cf46c9e7fc2b2452be WatchSource:0}: Error finding container 5c23ac0445ebaff5300340da6613a5c0d2788876c0da44cf46c9e7fc2b2452be: Status 404 returned error can't find the container with id 5c23ac0445ebaff5300340da6613a5c0d2788876c0da44cf46c9e7fc2b2452be Sep 17 16:29:40.462258 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:40.462203 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" event={"ID":"1d3edbeb-6c3d-4098-ac79-3820ea076531","Type":"ContainerStarted","Data":"83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a"} Sep 17 16:29:40.462650 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:40.462277 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" event={"ID":"1d3edbeb-6c3d-4098-ac79-3820ea076531","Type":"ContainerStarted","Data":"5c23ac0445ebaff5300340da6613a5c0d2788876c0da44cf46c9e7fc2b2452be"} Sep 17 16:29:40.462650 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:40.462348 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:29:40.491122 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:40.491079 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" podStartSLOduration=1.4910630870000001 podStartE2EDuration="1.491063087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-17 16:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:29:40.48969099 +0000 UTC m=+2037.592936371" watchObservedRunningTime="2026-09-17 16:29:40.491063087 +0000 UTC m=+2037.594308521" Sep 17 16:29:46.471354 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:29:46.471324 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:30:43.409752 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:30:43.409713 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:30:43.412818 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:30:43.412791 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:30:43.419506 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:30:43.419483 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:30:43.422290 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:30:43.422270 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:30:43.422690 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:30:43.422670 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:30:43.425017 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:30:43.425002 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:35:43.437089 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:35:43.437058 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:35:43.439368 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:35:43.439347 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:35:43.444597 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:35:43.444571 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:35:43.446356 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:35:43.446340 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:35:43.447116 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:35:43.447089 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:35:43.448694 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:35:43.448678 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:35:51.145879 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:35:51.145843 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:35:52.983136 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:35:52.983102 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:36:51.353694 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:36:51.353630 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:37:53.812082 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:37:53.812048 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr"] Sep 17 16:37:53.812551 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:37:53.812267 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" podUID="1d3edbeb-6c3d-4098-ac79-3820ea076531" containerName="splitter-graph-6d5c6" containerID="cri-o://83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a" gracePeriod=30 Sep 17 16:37:53.853998 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:37:53.853971 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:37:54.076719 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:37:54.076628 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:37:54.164118 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:37:54.164087 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:37:56.469570 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:37:56.469533 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" podUID="1d3edbeb-6c3d-4098-ac79-3820ea076531" containerName="splitter-graph-6d5c6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:38:01.470486 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:01.470442 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" podUID="1d3edbeb-6c3d-4098-ac79-3820ea076531" containerName="splitter-graph-6d5c6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:38:06.470484 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:06.470437 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" podUID="1d3edbeb-6c3d-4098-ac79-3820ea076531" containerName="splitter-graph-6d5c6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:38:06.470870 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:06.470577 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:38:11.470466 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:11.470423 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" podUID="1d3edbeb-6c3d-4098-ac79-3820ea076531" containerName="splitter-graph-6d5c6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:38:16.470201 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:16.470158 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" podUID="1d3edbeb-6c3d-4098-ac79-3820ea076531" containerName="splitter-graph-6d5c6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:38:21.470188 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:21.470145 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" podUID="1d3edbeb-6c3d-4098-ac79-3820ea076531" containerName="splitter-graph-6d5c6" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:38:23.995750 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:23.995725 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:38:24.012578 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.012553 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1d3edbeb-6c3d-4098-ac79-3820ea076531-proxy-tls\") pod \"1d3edbeb-6c3d-4098-ac79-3820ea076531\" (UID: \"1d3edbeb-6c3d-4098-ac79-3820ea076531\") " Sep 17 16:38:24.012698 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.012589 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d3edbeb-6c3d-4098-ac79-3820ea076531-openshift-service-ca-bundle\") pod \"1d3edbeb-6c3d-4098-ac79-3820ea076531\" (UID: \"1d3edbeb-6c3d-4098-ac79-3820ea076531\") " Sep 17 16:38:24.012965 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.012942 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d3edbeb-6c3d-4098-ac79-3820ea076531-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "1d3edbeb-6c3d-4098-ac79-3820ea076531" (UID: "1d3edbeb-6c3d-4098-ac79-3820ea076531"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 16:38:24.014649 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.014618 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d3edbeb-6c3d-4098-ac79-3820ea076531-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1d3edbeb-6c3d-4098-ac79-3820ea076531" (UID: "1d3edbeb-6c3d-4098-ac79-3820ea076531"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 16:38:24.114112 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.114031 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1d3edbeb-6c3d-4098-ac79-3820ea076531-proxy-tls\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:38:24.114112 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.114058 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1d3edbeb-6c3d-4098-ac79-3820ea076531-openshift-service-ca-bundle\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:38:24.472841 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.472759 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d3edbeb-6c3d-4098-ac79-3820ea076531" containerID="83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a" exitCode=0 Sep 17 16:38:24.472841 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.472804 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" Sep 17 16:38:24.472841 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.472817 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" event={"ID":"1d3edbeb-6c3d-4098-ac79-3820ea076531","Type":"ContainerDied","Data":"83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a"} Sep 17 16:38:24.473067 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.472858 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr" event={"ID":"1d3edbeb-6c3d-4098-ac79-3820ea076531","Type":"ContainerDied","Data":"5c23ac0445ebaff5300340da6613a5c0d2788876c0da44cf46c9e7fc2b2452be"} Sep 17 16:38:24.473067 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.472883 2580 scope.go:117] "RemoveContainer" containerID="83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a" Sep 17 16:38:24.481249 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.481232 2580 scope.go:117] "RemoveContainer" containerID="83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a" Sep 17 16:38:24.481506 ip-10-0-128-222 kubenswrapper[2580]: E0917 16:38:24.481483 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a\": container with ID starting with 83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a not found: ID does not exist" containerID="83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a" Sep 17 16:38:24.481569 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.481515 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a"} err="failed to get container status \"83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a\": rpc error: code = NotFound desc = could not find container \"83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a\": container with ID starting with 83200d5cbeff7d355397970597acdeb3e95f9371160d6c9b0437c278c65e2e6a not found: ID does not exist" Sep 17 16:38:24.512917 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.512844 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr"] Sep 17 16:38:24.515360 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:24.515338 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-6d5c6-5d4986d579-ngwcr"] Sep 17 16:38:25.491886 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:38:25.491847 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d3edbeb-6c3d-4098-ac79-3820ea076531" path="/var/lib/kubelet/pods/1d3edbeb-6c3d-4098-ac79-3820ea076531/volumes" Sep 17 16:39:03.676755 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:39:03.676716 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:40:43.461050 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:40:43.460983 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:40:43.463082 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:40:43.463059 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:40:43.468433 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:40:43.468418 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:40:43.470168 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:40:43.470151 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:40:43.471093 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:40:43.471077 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:40:43.472660 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:40:43.472644 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:44:49.901559 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:44:49.901527 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:45:28.137719 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:28.137629 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25"] Sep 17 16:45:28.138310 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:28.137899 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" podUID="28ef2274-6b65-469d-9491-19ccb33a1564" containerName="switch-graph-37c70" containerID="cri-o://b4220e051911a959dcaaba80761f691d667a61a1ea5412b3523fafbd334b3986" gracePeriod=30 Sep 17 16:45:28.443687 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:28.443597 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:45:28.558180 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:28.558147 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:45:29.949232 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:29.949195 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:45:30.044881 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:30.044843 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:45:31.352448 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:31.352407 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" podUID="28ef2274-6b65-469d-9491-19ccb33a1564" containerName="switch-graph-37c70" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:45:36.352482 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:36.352444 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" podUID="28ef2274-6b65-469d-9491-19ccb33a1564" containerName="switch-graph-37c70" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:45:41.352349 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:41.352310 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" podUID="28ef2274-6b65-469d-9491-19ccb33a1564" containerName="switch-graph-37c70" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:45:41.352734 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:41.352469 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:45:43.485832 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:43.485805 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:45:43.487710 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:43.487686 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:45:43.494229 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:43.494199 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:45:43.495671 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:43.495649 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:45:43.496837 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:43.496820 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:45:43.498061 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:43.498043 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:45:46.062001 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:46.061970 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:46.351621 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:46.351533 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" podUID="28ef2274-6b65-469d-9491-19ccb33a1564" containerName="switch-graph-37c70" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:45:46.932505 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:46.932478 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:47.798023 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:47.797995 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:48.605941 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:48.605909 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:49.410271 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:49.410241 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:50.190956 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:50.190930 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:50.971000 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:50.970940 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:51.352336 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:51.352302 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" podUID="28ef2274-6b65-469d-9491-19ccb33a1564" containerName="switch-graph-37c70" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:45:51.774719 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:51.774627 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:52.556987 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:52.556956 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:53.352133 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:53.352109 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:54.137476 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:54.137445 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:54.952828 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:54.952794 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:55.777548 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:55.777506 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:56.351905 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:56.351871 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" podUID="28ef2274-6b65-469d-9491-19ccb33a1564" containerName="switch-graph-37c70" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 17 16:45:56.614322 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:56.614236 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:57.492569 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:57.492527 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-37c70-86c98d65b-tnx25_28ef2274-6b65-469d-9491-19ccb33a1564/switch-graph-37c70/0.log" Sep 17 16:45:58.188377 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:58.188342 2580 generic.go:356] "Generic (PLEG): container finished" podID="28ef2274-6b65-469d-9491-19ccb33a1564" containerID="b4220e051911a959dcaaba80761f691d667a61a1ea5412b3523fafbd334b3986" exitCode=0 Sep 17 16:45:58.188557 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:58.188419 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" event={"ID":"28ef2274-6b65-469d-9491-19ccb33a1564","Type":"ContainerDied","Data":"b4220e051911a959dcaaba80761f691d667a61a1ea5412b3523fafbd334b3986"} Sep 17 16:45:58.283399 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:58.283376 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:45:58.321863 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:58.321830 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ef2274-6b65-469d-9491-19ccb33a1564-openshift-service-ca-bundle\") pod \"28ef2274-6b65-469d-9491-19ccb33a1564\" (UID: \"28ef2274-6b65-469d-9491-19ccb33a1564\") " Sep 17 16:45:58.322027 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:58.321893 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28ef2274-6b65-469d-9491-19ccb33a1564-proxy-tls\") pod \"28ef2274-6b65-469d-9491-19ccb33a1564\" (UID: \"28ef2274-6b65-469d-9491-19ccb33a1564\") " Sep 17 16:45:58.322275 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:58.322243 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28ef2274-6b65-469d-9491-19ccb33a1564-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "28ef2274-6b65-469d-9491-19ccb33a1564" (UID: "28ef2274-6b65-469d-9491-19ccb33a1564"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 17 16:45:58.324037 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:58.324013 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28ef2274-6b65-469d-9491-19ccb33a1564-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "28ef2274-6b65-469d-9491-19ccb33a1564" (UID: "28ef2274-6b65-469d-9491-19ccb33a1564"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 17 16:45:58.422618 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:58.422532 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28ef2274-6b65-469d-9491-19ccb33a1564-openshift-service-ca-bundle\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:45:58.422618 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:58.422564 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/28ef2274-6b65-469d-9491-19ccb33a1564-proxy-tls\") on node \"ip-10-0-128-222.ec2.internal\" DevicePath \"\"" Sep 17 16:45:59.193421 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:59.193390 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" Sep 17 16:45:59.193421 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:59.193407 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25" event={"ID":"28ef2274-6b65-469d-9491-19ccb33a1564","Type":"ContainerDied","Data":"536c011a488ebbad7f81d12076a0ba0cfc8c73200cd576e9246d22d99dd9361f"} Sep 17 16:45:59.194018 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:59.193473 2580 scope.go:117] "RemoveContainer" containerID="b4220e051911a959dcaaba80761f691d667a61a1ea5412b3523fafbd334b3986" Sep 17 16:45:59.216399 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:59.216377 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25"] Sep 17 16:45:59.223625 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:59.223605 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-37c70-86c98d65b-tnx25"] Sep 17 16:45:59.492981 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:59.492894 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28ef2274-6b65-469d-9491-19ccb33a1564" path="/var/lib/kubelet/pods/28ef2274-6b65-469d-9491-19ccb33a1564/volumes" Sep 17 16:45:59.670674 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:59.670643 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:45:59.856589 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:45:59.856556 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:46:00.136377 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:00.136284 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:46:03.033500 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:03.033461 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8877v_609285b3-0b12-4a2c-a723-afb1861e75a4/global-pull-secret-syncer/0.log" Sep 17 16:46:03.185923 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:03.185886 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" Sep 17 16:46:03.287190 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:03.287108 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-222.ec2.internal_246f67986432aefd3f131bae29e7bad0/haproxy/0.log" Sep 17 16:46:05.926406 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:05.926372 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:46:05.994087 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:05.994050 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8sb8r_2be6f78e-ee45-40c6-ba36-de1d778ff97f/konnectivity-agent/0.log" limit=1024 Sep 17 16:46:07.372709 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:07.372670 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-847b8bd8d7-5fpms_9ac10347-81ed-4b23-8eac-ddc2a4c1d171/cluster-monitoring-operator/0.log" Sep 17 16:46:07.688520 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:07.688435 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fp6xb_b037dd9f-6811-40ad-9077-b085f2046267/node-exporter/0.log" Sep 17 16:46:07.724517 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:07.724486 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fp6xb_b037dd9f-6811-40ad-9077-b085f2046267/kube-rbac-proxy/0.log" Sep 17 16:46:07.757245 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:07.757203 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fp6xb_b037dd9f-6811-40ad-9077-b085f2046267/init-textfile/0.log" Sep 17 16:46:09.860375 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:09.860332 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-ghjkd_51b96393-ed39-45b3-baca-b0acb42c254b/networking-console-plugin/0.log" Sep 17 16:46:10.399817 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:10.399782 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/1.log" Sep 17 16:46:10.407511 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:10.407474 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-fd4bb46c-qp4q9_0a3b84c4-ac8d-4e8a-a606-aba79b7fe148/console-operator/2.log" Sep 17 16:46:11.402800 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.402773 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-5lzg5_15601992-7dfc-4244-848a-eccecf6bc1c5/volume-data-source-validator/0.log" Sep 17 16:46:11.521678 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.521645 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4"] Sep 17 16:46:11.521956 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.521942 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d3edbeb-6c3d-4098-ac79-3820ea076531" containerName="splitter-graph-6d5c6" Sep 17 16:46:11.522015 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.521960 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d3edbeb-6c3d-4098-ac79-3820ea076531" containerName="splitter-graph-6d5c6" Sep 17 16:46:11.522015 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.521986 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28ef2274-6b65-469d-9491-19ccb33a1564" containerName="switch-graph-37c70" Sep 17 16:46:11.522015 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.521992 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="28ef2274-6b65-469d-9491-19ccb33a1564" containerName="switch-graph-37c70" Sep 17 16:46:11.522109 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.522049 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="28ef2274-6b65-469d-9491-19ccb33a1564" containerName="switch-graph-37c70" Sep 17 16:46:11.522109 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.522058 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d3edbeb-6c3d-4098-ac79-3820ea076531" containerName="splitter-graph-6d5c6" Sep 17 16:46:11.526406 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.526388 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.529109 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.529085 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pmzkb\"/\"openshift-service-ca.crt\"" Sep 17 16:46:11.529251 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.529142 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pmzkb\"/\"default-dockercfg-7gfcx\"" Sep 17 16:46:11.529251 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.529142 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pmzkb\"/\"kube-root-ca.crt\"" Sep 17 16:46:11.534552 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.534529 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4"] Sep 17 16:46:11.626300 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.626262 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-sys\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.626485 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.626319 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-lib-modules\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.626485 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.626384 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7djv8\" (UniqueName: \"kubernetes.io/projected/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-kube-api-access-7djv8\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.626485 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.626446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-podres\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.626485 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.626469 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-proc\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.727599 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.727506 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-podres\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.727599 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.727548 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-proc\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.727599 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.727577 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-sys\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.727844 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.727645 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-sys\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.727844 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.727689 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-proc\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.727844 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.727704 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-podres\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.727844 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.727728 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-lib-modules\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.727844 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.727751 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7djv8\" (UniqueName: \"kubernetes.io/projected/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-kube-api-access-7djv8\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.728007 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.727878 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-lib-modules\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.737070 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.737040 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7djv8\" (UniqueName: \"kubernetes.io/projected/9dbb4c21-c627-41bb-ada4-8d198fb52cb7-kube-api-access-7djv8\") pod \"perf-node-gather-daemonset-cdkk4\" (UID: \"9dbb4c21-c627-41bb-ada4-8d198fb52cb7\") " pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:11.837086 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:11.837047 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:12.105592 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:12.105560 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4"] Sep 17 16:46:12.108882 ip-10-0-128-222 kubenswrapper[2580]: W0917 16:46:12.108858 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9dbb4c21_c627_41bb_ada4_8d198fb52cb7.slice/crio-84d4619fd76563f02e0815bb1438bdd1f950fe22883082d9f471d42db00929bc WatchSource:0}: Error finding container 84d4619fd76563f02e0815bb1438bdd1f950fe22883082d9f471d42db00929bc: Status 404 returned error can't find the container with id 84d4619fd76563f02e0815bb1438bdd1f950fe22883082d9f471d42db00929bc Sep 17 16:46:12.250621 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:12.250586 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" event={"ID":"9dbb4c21-c627-41bb-ada4-8d198fb52cb7","Type":"ContainerStarted","Data":"706e3fa673e7681f12050679f1b2dd01de3d829f0f7328cfd3529c8f315f2417"} Sep 17 16:46:12.250787 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:12.250639 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" event={"ID":"9dbb4c21-c627-41bb-ada4-8d198fb52cb7","Type":"ContainerStarted","Data":"84d4619fd76563f02e0815bb1438bdd1f950fe22883082d9f471d42db00929bc"} Sep 17 16:46:12.250787 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:12.250670 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:12.271466 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:12.271416 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" podStartSLOduration=1.271398035 podStartE2EDuration="1.271398035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-17 16:46:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-17 16:46:12.270237444 +0000 UTC m=+3029.373482821" watchObservedRunningTime="2026-09-17 16:46:12.271398035 +0000 UTC m=+3029.374643464" Sep 17 16:46:12.342621 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:12.342593 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2bdcw_d24009de-229f-4bb3-a49e-5499cfc0b519/dns/0.log" Sep 17 16:46:12.367054 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:12.366982 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2bdcw_d24009de-229f-4bb3-a49e-5499cfc0b519/kube-rbac-proxy/0.log" Sep 17 16:46:12.533338 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:12.533306 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fj294_3c7b711c-7da3-4f63-ade9-9282296f6086/dns-node-resolver/0.log" Sep 17 16:46:13.128093 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:13.128066 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vlcft_edf5c407-8316-462e-b452-f867f1f5f301/node-ca/0.log" Sep 17 16:46:13.920301 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:13.920269 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-86dfcf9bc6-7j9bp_99684b4e-9484-4867-a730-ea1662218945/router/0.log" Sep 17 16:46:14.321978 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:14.321945 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-88zv4_5f94a7e8-2cf1-4051-a1d5-c7d29241fab3/serve-healthcheck-canary/0.log" Sep 17 16:46:14.749252 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:14.749125 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-wps4t_fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b/insights-operator/0.log" Sep 17 16:46:14.750544 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:14.750524 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-wps4t_fd31c961-38dd-4bc2-b5e7-f6e1cf778a2b/insights-operator/1.log" Sep 17 16:46:14.928446 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:14.928424 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6fskk_1b832eb9-54df-42b4-8e08-a3f28c1f598e/kube-rbac-proxy/0.log" Sep 17 16:46:14.953313 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:14.953276 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6fskk_1b832eb9-54df-42b4-8e08-a3f28c1f598e/exporter/0.log" Sep 17 16:46:14.982087 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:14.982058 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6fskk_1b832eb9-54df-42b4-8e08-a3f28c1f598e/extractor/0.log" Sep 17 16:46:18.264300 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:18.264272 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pmzkb/perf-node-gather-daemonset-cdkk4" Sep 17 16:46:22.795056 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:22.795024 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-lh7sh_4861c6ad-a4a3-4947-a1f1-b21601e7a455/kube-storage-version-migrator-operator/1.log" Sep 17 16:46:22.796698 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:22.796675 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-lh7sh_4861c6ad-a4a3-4947-a1f1-b21601e7a455/kube-storage-version-migrator-operator/0.log" Sep 17 16:46:23.883565 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:23.883531 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/0.log" Sep 17 16:46:23.894562 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:23.894534 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2slv4_cb4be840-5ccd-4ea0-900c-ee3ad4111e15/kube-multus/1.log" Sep 17 16:46:24.268562 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:24.268480 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sdkh6_87ffca99-52b2-48df-bd31-35847f0524e6/kube-multus-additional-cni-plugins/0.log" Sep 17 16:46:24.323277 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:24.323241 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sdkh6_87ffca99-52b2-48df-bd31-35847f0524e6/egress-router-binary-copy/0.log" Sep 17 16:46:24.358202 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:24.358171 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sdkh6_87ffca99-52b2-48df-bd31-35847f0524e6/cni-plugins/0.log" Sep 17 16:46:24.388329 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:24.388297 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sdkh6_87ffca99-52b2-48df-bd31-35847f0524e6/bond-cni-plugin/0.log" Sep 17 16:46:24.422350 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:24.422316 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sdkh6_87ffca99-52b2-48df-bd31-35847f0524e6/routeoverride-cni/0.log" Sep 17 16:46:24.463392 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:24.463365 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sdkh6_87ffca99-52b2-48df-bd31-35847f0524e6/whereabouts-cni-bincopy/0.log" Sep 17 16:46:24.523070 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:24.522999 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-sdkh6_87ffca99-52b2-48df-bd31-35847f0524e6/whereabouts-cni/0.log" Sep 17 16:46:25.441331 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:25.441286 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8clfl_05ed94c8-e908-40bc-abf2-b770ebc1344a/network-metrics-daemon/0.log" Sep 17 16:46:25.470274 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:25.470243 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8clfl_05ed94c8-e908-40bc-abf2-b770ebc1344a/kube-rbac-proxy/0.log" Sep 17 16:46:26.369833 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:26.369802 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-controller/0.log" Sep 17 16:46:26.392059 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:26.392034 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/0.log" Sep 17 16:46:26.392255 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:26.392069 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovn-acl-logging/1.log" Sep 17 16:46:26.421639 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:26.421608 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/kube-rbac-proxy-node/0.log" Sep 17 16:46:26.464700 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:26.464669 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/kube-rbac-proxy-ovn-metrics/0.log" Sep 17 16:46:26.496271 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:26.496247 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/northd/0.log" Sep 17 16:46:26.525611 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:26.525585 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/nbdb/0.log" Sep 17 16:46:26.560541 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:26.560512 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/sbdb/0.log" Sep 17 16:46:26.639936 ip-10-0-128-222 kubenswrapper[2580]: I0917 16:46:26.639871 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7zb8p_e7d7c2f1-e491-428d-98e5-573bb6cc97d0/ovnkube-controller/0.log"