Sep 16 18:38:26.416336 ip-10-0-133-99 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 16 18:38:26.416346 ip-10-0-133-99 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 16 18:38:26.416355 ip-10-0-133-99 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 16 18:38:26.416542 ip-10-0-133-99 systemd[1]: Failed to start Kubernetes Kubelet. Sep 16 18:38:36.530001 ip-10-0-133-99 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 16 18:38:36.530022 ip-10-0-133-99 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 3c0eed88e7ec4152b96f1f95ea15ae60 -- Sep 16 18:40:14.208513 ip-10-0-133-99 systemd[1]: Starting Kubernetes Kubelet... Sep 16 18:40:14.579823 ip-10-0-133-99 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 16 18:40:14.579823 ip-10-0-133-99 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 16 18:40:14.579823 ip-10-0-133-99 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 16 18:40:14.579823 ip-10-0-133-99 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 16 18:40:14.579823 ip-10-0-133-99 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 16 18:40:14.581651 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.581566 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 16 18:40:14.586683 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586666 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 16 18:40:14.586683 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586684 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586688 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586691 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586695 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586698 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586701 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586704 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586706 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586711 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586715 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586718 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586721 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586724 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586727 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586737 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586740 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586743 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586745 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586748 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 16 18:40:14.586749 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586751 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586753 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586756 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586759 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586762 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586765 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586768 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586771 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586773 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586776 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586778 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586781 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586783 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586786 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586788 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586790 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586793 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586796 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586799 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586802 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 16 18:40:14.587218 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586804 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586807 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586810 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586812 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586814 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586817 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586819 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586822 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586824 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586827 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586829 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586831 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586834 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586836 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586840 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586843 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586845 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586848 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586851 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586853 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 16 18:40:14.587747 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586856 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586858 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586861 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586864 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586866 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586869 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586873 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586877 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586880 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586883 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586885 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586889 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586891 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586894 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586897 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586899 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586902 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586904 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586906 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 16 18:40:14.588269 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586909 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586911 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586914 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586916 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586919 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586921 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586924 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586927 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586930 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586932 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586935 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586938 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586940 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586942 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586945 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586947 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586950 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586953 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586957 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.586960 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 16 18:40:14.588726 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587649 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587656 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587659 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587662 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587665 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587669 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587672 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587675 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587677 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587680 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587683 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587686 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587688 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587691 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587694 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587697 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587699 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587702 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587706 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587709 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 16 18:40:14.589243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587712 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587714 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587717 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587720 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587722 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587725 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587727 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587730 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587734 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587738 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587741 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587744 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587747 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587750 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587754 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587757 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587760 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587763 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587767 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587771 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 16 18:40:14.589834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587774 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587777 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587780 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587784 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587787 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587789 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587792 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587795 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587797 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587800 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587802 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587805 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587808 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587811 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587813 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587816 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587819 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587821 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587824 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587826 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 16 18:40:14.590384 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587828 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587831 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587834 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587836 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587839 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587841 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587844 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587846 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587849 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587852 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587855 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587858 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587860 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587863 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587866 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587868 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587872 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587875 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587877 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587880 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 16 18:40:14.590881 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587882 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587885 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587888 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587891 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587894 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587897 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587899 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587903 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587907 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587911 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587915 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587918 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587921 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587924 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587927 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587929 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587932 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587935 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.587937 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 16 18:40:14.591382 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588339 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588346 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588350 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588354 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588358 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588361 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588364 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588367 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588369 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588373 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588375 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588378 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588381 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588383 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588386 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588388 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588391 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588395 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588398 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 16 18:40:14.591854 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588401 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588404 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588407 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588410 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588413 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588416 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588418 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588421 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588423 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588426 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588428 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588431 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588433 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588436 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588439 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588442 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588445 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588447 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588450 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588453 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 16 18:40:14.592322 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588455 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588458 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588462 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588465 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588468 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588470 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588473 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588475 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588478 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588480 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588483 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588485 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588488 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588490 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588493 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588495 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588498 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588509 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588512 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588515 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 16 18:40:14.592816 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588518 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588520 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588523 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588526 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588528 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588531 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588536 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588539 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588542 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588545 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588548 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588551 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588554 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588557 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588560 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588563 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588566 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588570 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588572 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 16 18:40:14.593335 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588575 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588578 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588580 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588582 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588585 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588587 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588590 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588593 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588595 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588597 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588600 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588603 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588605 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588607 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588610 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588612 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588615 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588617 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588622 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 16 18:40:14.593807 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588624 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.588627 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588697 2566 flags.go:64] FLAG: --address="0.0.0.0" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588732 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588739 2566 flags.go:64] FLAG: --anonymous-auth="true" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588743 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588748 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588751 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588757 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588762 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588765 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588768 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588771 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588775 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588778 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588781 2566 flags.go:64] FLAG: --cgroup-root="" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588784 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588787 2566 flags.go:64] FLAG: --client-ca-file="" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588790 2566 flags.go:64] FLAG: --cloud-provider="external" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588792 2566 flags.go:64] FLAG: --cluster-dns="[]" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588797 2566 flags.go:64] FLAG: --cluster-domain="" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588799 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588803 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588806 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 16 18:40:14.594291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588809 2566 flags.go:64] FLAG: --container-log-max-files="5" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588813 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588816 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588819 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588822 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588825 2566 flags.go:64] FLAG: --contention-profiling="false" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588828 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588835 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588838 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588841 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588845 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588848 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588851 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588854 2566 flags.go:64] FLAG: --enable-load-reader="false" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588856 2566 flags.go:64] FLAG: --enable-server="true" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588860 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588865 2566 flags.go:64] FLAG: --event-burst="100" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588868 2566 flags.go:64] FLAG: --event-qps="50" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588871 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588874 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588877 2566 flags.go:64] FLAG: --eviction-hard="" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588881 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588884 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588887 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588890 2566 flags.go:64] FLAG: --eviction-soft="" Sep 16 18:40:14.594870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588892 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588895 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588899 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588902 2566 flags.go:64] FLAG: --experimental-mounter-path="" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588905 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588908 2566 flags.go:64] FLAG: --fail-swap-on="true" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588911 2566 flags.go:64] FLAG: --feature-gates="" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588914 2566 flags.go:64] FLAG: --file-check-frequency="20s" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588918 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588921 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588924 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588927 2566 flags.go:64] FLAG: --healthz-port="10248" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588930 2566 flags.go:64] FLAG: --help="false" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588933 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-133-99.ec2.internal" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588937 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588940 2566 flags.go:64] FLAG: --http-check-frequency="20s" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588943 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588947 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588950 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588953 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588956 2566 flags.go:64] FLAG: --image-service-endpoint="" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588958 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588962 2566 flags.go:64] FLAG: --kube-api-burst="100" Sep 16 18:40:14.595484 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588965 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588968 2566 flags.go:64] FLAG: --kube-api-qps="50" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588971 2566 flags.go:64] FLAG: --kube-reserved="" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588974 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588978 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588981 2566 flags.go:64] FLAG: --kubelet-cgroups="" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588983 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588986 2566 flags.go:64] FLAG: --lock-file="" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588989 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588992 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.588995 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589000 2566 flags.go:64] FLAG: --log-json-split-stream="false" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589003 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589006 2566 flags.go:64] FLAG: --log-text-split-stream="false" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589009 2566 flags.go:64] FLAG: --logging-format="text" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589011 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589015 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589018 2566 flags.go:64] FLAG: --manifest-url="" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589020 2566 flags.go:64] FLAG: --manifest-url-header="" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589025 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589028 2566 flags.go:64] FLAG: --max-open-files="1000000" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589031 2566 flags.go:64] FLAG: --max-pods="110" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589034 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589040 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589042 2566 flags.go:64] FLAG: --memory-manager-policy="None" Sep 16 18:40:14.596096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589045 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589048 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589051 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589054 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589061 2566 flags.go:64] FLAG: --node-status-max-images="50" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589065 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589068 2566 flags.go:64] FLAG: --oom-score-adj="-999" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589071 2566 flags.go:64] FLAG: --pod-cidr="" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589074 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589079 2566 flags.go:64] FLAG: --pod-manifest-path="" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589083 2566 flags.go:64] FLAG: --pod-max-pids="-1" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589086 2566 flags.go:64] FLAG: --pods-per-core="0" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589089 2566 flags.go:64] FLAG: --port="10250" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589092 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589095 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-092beffb8bbfb0eb3" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589098 2566 flags.go:64] FLAG: --qos-reserved="" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589102 2566 flags.go:64] FLAG: --read-only-port="10255" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589104 2566 flags.go:64] FLAG: --register-node="true" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589107 2566 flags.go:64] FLAG: --register-with-taints="" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589111 2566 flags.go:64] FLAG: --registry-burst="10" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589114 2566 flags.go:64] FLAG: --registry-qps="5" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589117 2566 flags.go:64] FLAG: --reserved-cpus="" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589119 2566 flags.go:64] FLAG: --reserved-memory="" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589123 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589126 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 16 18:40:14.597154 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589129 2566 flags.go:64] FLAG: --rotate-certificates="false" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589132 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589135 2566 flags.go:64] FLAG: --runonce="false" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589138 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589141 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589145 2566 flags.go:64] FLAG: --seccomp-default="false" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589149 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589152 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589155 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589158 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589161 2566 flags.go:64] FLAG: --storage-driver-password="root" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589164 2566 flags.go:64] FLAG: --storage-driver-secure="false" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589167 2566 flags.go:64] FLAG: --storage-driver-table="stats" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589170 2566 flags.go:64] FLAG: --storage-driver-user="root" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589172 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589175 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589179 2566 flags.go:64] FLAG: --system-cgroups="" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589182 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589201 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589204 2566 flags.go:64] FLAG: --tls-cert-file="" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589207 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589211 2566 flags.go:64] FLAG: --tls-min-version="" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589214 2566 flags.go:64] FLAG: --tls-private-key-file="" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589217 2566 flags.go:64] FLAG: --topology-manager-policy="none" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589219 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 16 18:40:14.598270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589222 2566 flags.go:64] FLAG: --topology-manager-scope="container" Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589225 2566 flags.go:64] FLAG: --v="2" Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589229 2566 flags.go:64] FLAG: --version="false" Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589233 2566 flags.go:64] FLAG: --vmodule="" Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589236 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.589239 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589355 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589359 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589362 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589365 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589367 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589370 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589374 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589378 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589381 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589383 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589386 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589389 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589391 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589394 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589396 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589399 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 16 18:40:14.599145 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589402 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589405 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589408 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589411 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589414 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589417 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589421 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589425 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589427 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589430 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589433 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589435 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589438 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589441 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589443 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589446 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589448 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589451 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589454 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 16 18:40:14.599779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589457 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589459 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589462 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589466 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589470 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589472 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589475 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589478 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589480 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589483 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589486 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589488 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589491 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589494 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589498 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589501 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589505 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589508 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589511 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 16 18:40:14.600631 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589513 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589516 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589519 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589521 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589524 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589526 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589529 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589531 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589534 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589536 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589539 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589541 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589544 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589546 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589549 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589552 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589555 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589561 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589564 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 16 18:40:14.601407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589567 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589570 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589573 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589575 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589578 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589580 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589583 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589585 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589588 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589591 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589593 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589597 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589599 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589602 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589604 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589607 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589609 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589612 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589615 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589617 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 16 18:40:14.601891 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589620 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589622 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589625 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589627 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589630 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.589632 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.590232 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.598780 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.598798 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.599606 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.599618 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599721 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599729 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599735 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599741 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 16 18:40:14.602698 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599747 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599752 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599756 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599761 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599765 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599769 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599774 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599777 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599789 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599794 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599802 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599807 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599812 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599816 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599821 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599825 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599829 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599833 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599837 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599842 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 16 18:40:14.603497 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599846 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599850 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599854 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599859 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599863 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599868 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599873 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599877 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599881 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599885 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599890 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599894 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599898 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599902 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599909 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599915 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599920 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599924 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599928 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599933 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 16 18:40:14.604052 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599937 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599949 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599954 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599959 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599963 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599967 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599971 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599976 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599980 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599984 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599988 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599992 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.599996 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600000 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600005 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600009 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600013 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600020 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600026 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 16 18:40:14.604711 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600031 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600035 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600041 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600045 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600050 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600055 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600059 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600064 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600069 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600073 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600079 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600083 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600088 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600093 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600097 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600109 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600114 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600118 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600122 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 16 18:40:14.605243 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600126 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600130 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600135 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600139 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600143 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600147 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600151 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600155 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600159 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600163 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600167 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600173 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600178 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600182 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600205 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600210 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 16 18:40:14.605764 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600214 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.600223 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600457 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600465 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600470 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600475 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600479 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600483 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600488 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600492 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600496 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600500 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600504 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600516 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600522 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 16 18:40:14.606282 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600530 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600534 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600538 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600542 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600547 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600551 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600555 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600559 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600563 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600568 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600572 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600577 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600581 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600585 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600589 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600593 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600598 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600602 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600606 2566 feature_gate.go:328] unrecognized feature gate: Example Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600613 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 16 18:40:14.606659 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600618 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600623 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600627 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600631 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600636 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600639 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600643 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600647 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600651 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600655 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600659 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600671 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600676 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600680 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600684 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600688 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600692 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600696 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600700 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 16 18:40:14.607168 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600704 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600708 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600712 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600719 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600725 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600729 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600734 2566 feature_gate.go:328] unrecognized feature gate: Example2 Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600739 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600743 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600747 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600751 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600756 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600760 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600764 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600768 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600772 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600776 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600780 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600784 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600788 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 16 18:40:14.607834 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600793 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600796 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600801 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600805 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600809 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600820 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600825 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600829 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600833 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600837 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600842 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600846 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600850 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600854 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600858 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600862 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600867 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600871 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600876 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 16 18:40:14.608542 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600879 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600884 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600888 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600892 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600896 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600900 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600904 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:14.600908 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.600916 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.601084 2566 server.go:962] "Client rotation is on, will bootstrap in background" Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.604240 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.605150 2566 server.go:1019] "Starting client certificate rotation" Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.605223 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 16 18:40:14.609028 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.605258 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 16 18:40:14.609386 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.608150 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 16 18:40:14.616379 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.616356 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 16 18:40:14.624098 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.624083 2566 log.go:25] "Validated CRI v1 runtime API" Sep 16 18:40:14.627210 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.627182 2566 log.go:25] "Validated CRI v1 image API" Sep 16 18:40:14.627981 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.627961 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 16 18:40:14.630281 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.630248 2566 fs.go:135] Filesystem UUIDs: map[379eeedb-f375-4b7d-91b8-0d24471f4ab4:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d935ee5f-4bf0-48c0-9d79-4c43f7950aa0:/dev/nvme0n1p3] Sep 16 18:40:14.630367 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.630279 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 16 18:40:14.635197 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.635063 2566 manager.go:216] Machine: {Timestamp:2026-09-16 18:40:14.633871378 +0000 UTC m=+0.314726001 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108664 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b5833502f6cd9fcfb44fb5d2bd85c SystemUUID:ec2b5833-502f-6cd9-fcfb-44fb5d2bd85c BootID:3c0eed88-e7ec-4152-b96f-1f95ea15ae60 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:cc:57:65:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:cc:57:65:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:54:cd:a9:e0:ad Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 16 18:40:14.635197 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.635176 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 16 18:40:14.635352 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.635295 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 16 18:40:14.637240 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.637214 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 16 18:40:14.637368 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.637242 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-99.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 16 18:40:14.638074 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.638064 2566 topology_manager.go:138] "Creating topology manager with none policy" Sep 16 18:40:14.638105 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.638076 2566 container_manager_linux.go:309] "Creating device plugin manager" Sep 16 18:40:14.638105 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.638084 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 16 18:40:14.638720 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.638709 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 16 18:40:14.639745 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.639735 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 16 18:40:14.643972 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.643960 2566 state_mem.go:36] "Initialized new in-memory state store" Sep 16 18:40:14.644075 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.644066 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 16 18:40:14.644112 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.644098 2566 kubelet.go:486] "Attempting to sync node with API server" Sep 16 18:40:14.644112 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.644108 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 16 18:40:14.644174 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.644122 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 16 18:40:14.644174 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.644134 2566 kubelet.go:398] "Adding apiserver pod source" Sep 16 18:40:14.644174 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.644142 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 16 18:40:14.644525 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.644513 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 16 18:40:14.644567 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.644532 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 16 18:40:14.646563 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.646547 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 16 18:40:14.647287 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.647270 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 16 18:40:14.649116 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.649103 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 16 18:40:14.649162 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.649133 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 16 18:40:14.650849 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.650837 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 16 18:40:14.650895 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.650856 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 16 18:40:14.650895 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.650862 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 16 18:40:14.650895 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.650868 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 16 18:40:14.650895 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.650876 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 16 18:40:14.650895 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.650882 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 16 18:40:14.650895 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.650897 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 16 18:40:14.651056 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.650905 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 16 18:40:14.651056 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.650934 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 16 18:40:14.651056 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.650944 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 16 18:40:14.651056 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.650963 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 16 18:40:14.651056 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.650971 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 16 18:40:14.651922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.651881 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 16 18:40:14.652035 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.652024 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 16 18:40:14.654896 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.654871 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-99.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 16 18:40:14.654975 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.654913 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 16 18:40:14.654975 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.654964 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 16 18:40:14.656137 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.656124 2566 server.go:1268] "Started kubelet" Sep 16 18:40:14.656235 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.656209 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 16 18:40:14.656350 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.656205 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 16 18:40:14.656430 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.656399 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 16 18:40:14.656852 ip-10-0-133-99 systemd[1]: Started Kubernetes Kubelet. Sep 16 18:40:14.658307 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.658173 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 16 18:40:14.658307 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.658215 2566 server.go:310] "Adding debug handlers to kubelet server" Sep 16 18:40:14.661348 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.661329 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 16 18:40:14.664023 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.664009 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 16 18:40:14.664511 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.664494 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 16 18:40:14.664663 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.664639 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-99.ec2.internal\" not found" Sep 16 18:40:14.665246 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665229 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 16 18:40:14.665246 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665230 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 16 18:40:14.665364 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665260 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 16 18:40:14.665364 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665305 2566 factory.go:55] Registering systemd factory Sep 16 18:40:14.665364 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665321 2566 factory.go:223] Registration of the systemd container factory successfully Sep 16 18:40:14.665524 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665376 2566 reconstruct.go:97] "Volume reconstruction finished" Sep 16 18:40:14.665524 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665384 2566 reconciler.go:29] "Reconciler: start to sync state" Sep 16 18:40:14.665613 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665534 2566 factory.go:153] Registering CRI-O factory Sep 16 18:40:14.665613 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665547 2566 factory.go:223] Registration of the crio container factory successfully Sep 16 18:40:14.665613 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665589 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 16 18:40:14.665613 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665602 2566 factory.go:103] Registering Raw factory Sep 16 18:40:14.665613 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665615 2566 manager.go:1196] Started watching for new ooms in manager Sep 16 18:40:14.665952 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.665936 2566 manager.go:349] Starting recovery of all containers Sep 16 18:40:14.666212 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.665165 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-99.ec2.internal.18d5e12e014b521f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-99.ec2.internal,UID:ip-10-0-133-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-99.ec2.internal,},FirstTimestamp:2026-09-16 18:40:14.656098847 +0000 UTC m=+0.336953470,LastTimestamp:2026-09-16 18:40:14.656098847 +0000 UTC m=+0.336953470,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-99.ec2.internal,}" Sep 16 18:40:14.673506 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.673385 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 16 18:40:14.673506 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.673407 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-99.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 16 18:40:14.677421 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.677399 2566 manager.go:354] Recovery completed Sep 16 18:40:14.683071 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.683054 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 18:40:14.684908 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.684892 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientMemory" Sep 16 18:40:14.684981 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.684923 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 18:40:14.684981 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.684937 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientPID" Sep 16 18:40:14.685491 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.685421 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 16 18:40:14.685491 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.685432 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 16 18:40:14.685491 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.685440 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 18:40:14.685491 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.685448 2566 state_mem.go:36] "Initialized new in-memory state store" Sep 16 18:40:14.686129 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.686112 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientMemory" Sep 16 18:40:14.686167 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.686146 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 18:40:14.686167 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.686157 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientPID" Sep 16 18:40:14.686928 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.686862 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-99.ec2.internal.18d5e12e0302e8dc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-99.ec2.internal,UID:ip-10-0-133-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-99.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-99.ec2.internal,},FirstTimestamp:2026-09-16 18:40:14.68490774 +0000 UTC m=+0.365762366,LastTimestamp:2026-09-16 18:40:14.68490774 +0000 UTC m=+0.365762366,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-99.ec2.internal,}" Sep 16 18:40:14.687640 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.687624 2566 policy_none.go:49] "None policy: Start" Sep 16 18:40:14.687640 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.687641 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 16 18:40:14.687742 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.687651 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 16 18:40:14.689662 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.689652 2566 policy_none.go:47] "Start" Sep 16 18:40:14.701397 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.701336 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-99.ec2.internal.18d5e12e03033e3b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-99.ec2.internal,UID:ip-10-0-133-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-99.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-99.ec2.internal,},FirstTimestamp:2026-09-16 18:40:14.684929595 +0000 UTC m=+0.365784219,LastTimestamp:2026-09-16 18:40:14.684929595 +0000 UTC m=+0.365784219,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-99.ec2.internal,}" Sep 16 18:40:14.706999 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.706812 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-99.ec2.internal.18d5e12e030372f2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-99.ec2.internal,UID:ip-10-0-133-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-99.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-99.ec2.internal,},FirstTimestamp:2026-09-16 18:40:14.68494309 +0000 UTC m=+0.365797717,LastTimestamp:2026-09-16 18:40:14.68494309 +0000 UTC m=+0.365797717,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-99.ec2.internal,}" Sep 16 18:40:14.710751 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.710736 2566 manager.go:337] "Starting Device Plugin manager" Sep 16 18:40:14.710825 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.710780 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 16 18:40:14.710825 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.710792 2566 server.go:87] "Starting device plugin registration server" Sep 16 18:40:14.711094 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.711081 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 16 18:40:14.711157 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.711095 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 16 18:40:14.711385 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.711261 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 16 18:40:14.711385 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.711330 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 16 18:40:14.711385 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.711338 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 16 18:40:14.711942 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.711924 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 16 18:40:14.712003 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.711974 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-99.ec2.internal\" not found" Sep 16 18:40:14.713086 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.712750 2566 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-99.ec2.internal.18d5e12e0302e8dc\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-99.ec2.internal.18d5e12e0302e8dc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-99.ec2.internal,UID:ip-10-0-133-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-99.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-99.ec2.internal,},FirstTimestamp:2026-09-16 18:40:14.68490774 +0000 UTC m=+0.365762366,LastTimestamp:2026-09-16 18:40:14.686129249 +0000 UTC m=+0.366983875,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-99.ec2.internal,}" Sep 16 18:40:14.718006 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.717948 2566 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-99.ec2.internal.18d5e12e03033e3b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-99.ec2.internal.18d5e12e03033e3b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-99.ec2.internal,UID:ip-10-0-133-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-99.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-99.ec2.internal,},FirstTimestamp:2026-09-16 18:40:14.684929595 +0000 UTC m=+0.365784219,LastTimestamp:2026-09-16 18:40:14.686151672 +0000 UTC m=+0.367006295,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-99.ec2.internal,}" Sep 16 18:40:14.742363 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.725750 2566 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-133-99.ec2.internal.18d5e12e030372f2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-99.ec2.internal.18d5e12e030372f2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-99.ec2.internal,UID:ip-10-0-133-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-99.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-99.ec2.internal,},FirstTimestamp:2026-09-16 18:40:14.68494309 +0000 UTC m=+0.365797717,LastTimestamp:2026-09-16 18:40:14.686160579 +0000 UTC m=+0.367015203,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-99.ec2.internal,}" Sep 16 18:40:14.742363 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.735700 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-99.ec2.internal.18d5e12e0533db79 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-99.ec2.internal,UID:ip-10-0-133-99.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-133-99.ec2.internal,},FirstTimestamp:2026-09-16 18:40:14.721670009 +0000 UTC m=+0.402524621,LastTimestamp:2026-09-16 18:40:14.721670009 +0000 UTC m=+0.402524621,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-99.ec2.internal,}" Sep 16 18:40:14.789470 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.789446 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 16 18:40:14.790601 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.790584 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 16 18:40:14.790667 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.790634 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 16 18:40:14.790667 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.790657 2566 kubelet.go:2448] "Starting kubelet main sync loop" Sep 16 18:40:14.790762 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.790728 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 16 18:40:14.791838 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.791819 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xbcbk" Sep 16 18:40:14.800226 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.800208 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" Sep 16 18:40:14.802545 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.802532 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xbcbk" Sep 16 18:40:14.811859 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.811846 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 18:40:14.812607 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.812588 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientMemory" Sep 16 18:40:14.812687 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.812617 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 18:40:14.812687 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.812627 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientPID" Sep 16 18:40:14.812687 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.812648 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-99.ec2.internal" Sep 16 18:40:14.830782 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.830723 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-99.ec2.internal" Sep 16 18:40:14.830782 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.830743 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-99.ec2.internal\": node \"ip-10-0-133-99.ec2.internal\" not found" Sep 16 18:40:14.891138 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.891119 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-99.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal"] Sep 16 18:40:14.891253 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.891210 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 18:40:14.892581 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.892568 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientMemory" Sep 16 18:40:14.892655 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.892592 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 18:40:14.892655 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.892602 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientPID" Sep 16 18:40:14.894844 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.894832 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 18:40:14.894950 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.894935 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-99.ec2.internal" Sep 16 18:40:14.894990 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.894973 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 18:40:14.895514 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.895500 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientMemory" Sep 16 18:40:14.895596 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.895523 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 18:40:14.895596 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.895504 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientMemory" Sep 16 18:40:14.895596 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.895536 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientPID" Sep 16 18:40:14.895596 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.895552 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 18:40:14.895596 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.895563 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientPID" Sep 16 18:40:14.898342 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.898324 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" Sep 16 18:40:14.898409 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.898358 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 16 18:40:14.899016 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.899003 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientMemory" Sep 16 18:40:14.899086 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.899020 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasNoDiskPressure" Sep 16 18:40:14.899086 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.899030 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeHasSufficientPID" Sep 16 18:40:14.913698 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.913683 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-99.ec2.internal\" not found" node="ip-10-0-133-99.ec2.internal" Sep 16 18:40:14.918035 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.918020 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-99.ec2.internal\" not found" node="ip-10-0-133-99.ec2.internal" Sep 16 18:40:14.947323 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:14.947304 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-99.ec2.internal\" not found" Sep 16 18:40:14.966444 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.966426 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d5acb9fe33c38810a5100cf8c524f161-config\") pod \"kube-apiserver-proxy-ip-10-0-133-99.ec2.internal\" (UID: \"d5acb9fe33c38810a5100cf8c524f161\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-99.ec2.internal" Sep 16 18:40:14.966506 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.966449 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/53c18eb6ba51406b052ec7b9ec1cf464-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal\" (UID: \"53c18eb6ba51406b052ec7b9ec1cf464\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" Sep 16 18:40:14.966506 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:14.966466 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53c18eb6ba51406b052ec7b9ec1cf464-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal\" (UID: \"53c18eb6ba51406b052ec7b9ec1cf464\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" Sep 16 18:40:15.048070 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:15.048040 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-99.ec2.internal\" not found" Sep 16 18:40:15.066605 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.066578 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53c18eb6ba51406b052ec7b9ec1cf464-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal\" (UID: \"53c18eb6ba51406b052ec7b9ec1cf464\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" Sep 16 18:40:15.066726 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.066642 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d5acb9fe33c38810a5100cf8c524f161-config\") pod \"kube-apiserver-proxy-ip-10-0-133-99.ec2.internal\" (UID: \"d5acb9fe33c38810a5100cf8c524f161\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-99.ec2.internal" Sep 16 18:40:15.066726 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.066661 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d5acb9fe33c38810a5100cf8c524f161-config\") pod \"kube-apiserver-proxy-ip-10-0-133-99.ec2.internal\" (UID: \"d5acb9fe33c38810a5100cf8c524f161\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-99.ec2.internal" Sep 16 18:40:15.066726 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.066700 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/53c18eb6ba51406b052ec7b9ec1cf464-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal\" (UID: \"53c18eb6ba51406b052ec7b9ec1cf464\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" Sep 16 18:40:15.066726 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.066724 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/53c18eb6ba51406b052ec7b9ec1cf464-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal\" (UID: \"53c18eb6ba51406b052ec7b9ec1cf464\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" Sep 16 18:40:15.066863 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.066667 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53c18eb6ba51406b052ec7b9ec1cf464-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal\" (UID: \"53c18eb6ba51406b052ec7b9ec1cf464\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" Sep 16 18:40:15.149037 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:15.148977 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-99.ec2.internal\" not found" Sep 16 18:40:15.216368 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.216341 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-99.ec2.internal" Sep 16 18:40:15.221646 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.221554 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" Sep 16 18:40:15.249491 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:15.249469 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-99.ec2.internal\" not found" Sep 16 18:40:15.350427 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:15.350400 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-99.ec2.internal\" not found" Sep 16 18:40:15.456914 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:15.456859 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-99.ec2.internal\" not found" Sep 16 18:40:15.556967 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:15.556943 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-99.ec2.internal\" not found" Sep 16 18:40:15.604645 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.604616 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 16 18:40:15.658051 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:15.658028 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-99.ec2.internal\" not found" Sep 16 18:40:15.727528 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.727428 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 16 18:40:15.729776 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.729757 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 16 18:40:15.764703 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.764677 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-99.ec2.internal" Sep 16 18:40:15.776828 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.776810 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 16 18:40:15.778294 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.778280 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" Sep 16 18:40:15.798563 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.798548 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 16 18:40:15.806833 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.806807 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 18:35:15 +0000 UTC" deadline="2028-07-03 09:13:41.025269058 +0000 UTC" Sep 16 18:40:15.806934 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.806832 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15734h33m25.218439474s" Sep 16 18:40:15.817517 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.817496 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 16 18:40:15.927683 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:15.927644 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c18eb6ba51406b052ec7b9ec1cf464.slice/crio-066a9f7473250987a25888b2b93bc6c5865f8eece9b88eab5077a28d635f4991 WatchSource:0}: Error finding container 066a9f7473250987a25888b2b93bc6c5865f8eece9b88eab5077a28d635f4991: Status 404 returned error can't find the container with id 066a9f7473250987a25888b2b93bc6c5865f8eece9b88eab5077a28d635f4991 Sep 16 18:40:15.933561 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:15.933543 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 16 18:40:15.938974 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:15.938945 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5acb9fe33c38810a5100cf8c524f161.slice/crio-0c83f3418cc2b0905dfc48ead980bb0e0db132f416325c77ef19d07befd1361d WatchSource:0}: Error finding container 0c83f3418cc2b0905dfc48ead980bb0e0db132f416325c77ef19d07befd1361d: Status 404 returned error can't find the container with id 0c83f3418cc2b0905dfc48ead980bb0e0db132f416325c77ef19d07befd1361d Sep 16 18:40:16.040312 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.040292 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 16 18:40:16.645393 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.645361 2566 apiserver.go:52] "Watching apiserver" Sep 16 18:40:16.652387 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.652355 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 16 18:40:16.652873 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.652828 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-vxc8t","openshift-cluster-node-tuning-operator/tuned-fm6nk","openshift-dns/node-resolver-ldzq8","openshift-image-registry/node-ca-tjc8j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal","openshift-network-diagnostics/network-check-target-s4dw9","openshift-ovn-kubernetes/ovnkube-node-b55tw","kube-system/kube-apiserver-proxy-ip-10-0-133-99.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft","openshift-multus/multus-additional-cni-plugins-8fkdw","openshift-multus/multus-vkgn2","openshift-multus/network-metrics-daemon-c2scp","openshift-network-operator/iptables-alerter-m9wjt","kube-system/global-pull-secret-syncer-swmrz"] Sep 16 18:40:16.655681 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.655659 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.658451 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.658352 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.658768 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.658586 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x4cbm\"" Sep 16 18:40:16.660929 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.660750 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ldzq8" Sep 16 18:40:16.661423 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.661288 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 16 18:40:16.661423 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.661355 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 16 18:40:16.662204 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.662170 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 16 18:40:16.662626 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.662461 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 16 18:40:16.662626 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.662484 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-525qs\"" Sep 16 18:40:16.662779 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.662754 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 16 18:40:16.663381 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.663359 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 16 18:40:16.663482 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.663463 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 16 18:40:16.663600 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.663574 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 16 18:40:16.663792 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.663775 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 16 18:40:16.663851 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.663829 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m6kgg\"" Sep 16 18:40:16.664181 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.664009 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 16 18:40:16.664181 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.664059 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 16 18:40:16.664181 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.664112 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tjc8j" Sep 16 18:40:16.664512 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.664493 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 16 18:40:16.664601 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.664521 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 16 18:40:16.666720 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.666659 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:16.666787 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:16.666734 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:16.668242 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.667829 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 16 18:40:16.668242 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.667841 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 16 18:40:16.668242 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.668083 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zv5rc\"" Sep 16 18:40:16.668242 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.668112 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 16 18:40:16.669224 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.669197 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:16.671553 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:16.669280 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:16.672937 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.671829 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.674764 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.674740 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 16 18:40:16.675018 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.674998 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 16 18:40:16.675559 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.675534 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lnhvl\"" Sep 16 18:40:16.675815 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.675794 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 16 18:40:16.676311 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676290 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-multus-socket-dir-parent\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.676453 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676437 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd09345e-605b-4196-a5d3-1aa3e3010635-hosts-file\") pod \"node-resolver-ldzq8\" (UID: \"bd09345e-605b-4196-a5d3-1aa3e3010635\") " pod="openshift-dns/node-resolver-ldzq8" Sep 16 18:40:16.676575 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676556 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-systemd-units\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.676650 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676594 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/61d20eff-c5cd-43af-aa68-7ee018467a60-ovnkube-config\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.676650 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676628 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/61d20eff-c5cd-43af-aa68-7ee018467a60-ovn-node-metrics-cert\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.676761 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676661 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/61d20eff-c5cd-43af-aa68-7ee018467a60-ovnkube-script-lib\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.676761 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676695 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-multus-cni-dir\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.676761 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676743 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-var-lib-cni-multus\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.676911 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676841 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-var-lib-kubelet\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.676911 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676888 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-multus-daemon-config\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.677016 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676908 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-run-openvswitch\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.677016 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676945 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-run-ovn-kubernetes\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.677016 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.676973 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-run-netns\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.677016 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677002 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-run-multus-certs\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.677229 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677074 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-etc-kubernetes\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.677229 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677131 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw8mn\" (UniqueName: \"kubernetes.io/projected/bd09345e-605b-4196-a5d3-1aa3e3010635-kube-api-access-nw8mn\") pod \"node-resolver-ldzq8\" (UID: \"bd09345e-605b-4196-a5d3-1aa3e3010635\") " pod="openshift-dns/node-resolver-ldzq8" Sep 16 18:40:16.677229 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677167 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58075b65-6a9b-4ef3-9776-2cf7c3a5e15a-host\") pod \"node-ca-tjc8j\" (UID: \"58075b65-6a9b-4ef3-9776-2cf7c3a5e15a\") " pod="openshift-image-registry/node-ca-tjc8j" Sep 16 18:40:16.677229 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677196 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58075b65-6a9b-4ef3-9776-2cf7c3a5e15a-serviceca\") pod \"node-ca-tjc8j\" (UID: \"58075b65-6a9b-4ef3-9776-2cf7c3a5e15a\") " pod="openshift-image-registry/node-ca-tjc8j" Sep 16 18:40:16.677469 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677229 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-slash\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.677469 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677285 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-run-netns\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.677469 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677457 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-var-lib-openvswitch\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.677622 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677503 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-etc-openvswitch\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.677622 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677537 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lws2c\" (UniqueName: \"kubernetes.io/projected/61d20eff-c5cd-43af-aa68-7ee018467a60-kube-api-access-lws2c\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.677622 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677571 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-system-cni-dir\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.677622 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677599 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkqxs\" (UniqueName: \"kubernetes.io/projected/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-kube-api-access-gkqxs\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.677882 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677632 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thx9v\" (UniqueName: \"kubernetes.io/projected/58075b65-6a9b-4ef3-9776-2cf7c3a5e15a-kube-api-access-thx9v\") pod \"node-ca-tjc8j\" (UID: \"58075b65-6a9b-4ef3-9776-2cf7c3a5e15a\") " pod="openshift-image-registry/node-ca-tjc8j" Sep 16 18:40:16.677882 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677703 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-cni-bin\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.677882 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677873 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m9wjt" Sep 16 18:40:16.678033 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677896 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.678033 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677951 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-hostroot\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.678033 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.677992 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:16.678179 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678032 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-run-systemd\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.678179 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678065 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-run-k8s-cni-cncf-io\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.678179 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678100 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/664c47f5-9768-4395-8396-ce9ac30421e3-kubelet-config\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:16.678179 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678129 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-cni-netd\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.678179 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678160 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/61d20eff-c5cd-43af-aa68-7ee018467a60-env-overrides\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.678447 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678200 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-cni-binary-copy\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.678447 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678261 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-node-log\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.678447 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678300 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-log-socket\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.678447 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678361 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-cnibin\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.678711 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678686 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-var-lib-cni-bin\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.678773 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-multus-conf-dir\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.678773 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678762 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd09345e-605b-4196-a5d3-1aa3e3010635-tmp-dir\") pod \"node-resolver-ldzq8\" (UID: \"bd09345e-605b-4196-a5d3-1aa3e3010635\") " pod="openshift-dns/node-resolver-ldzq8" Sep 16 18:40:16.678881 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678785 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/664c47f5-9768-4395-8396-ce9ac30421e3-dbus\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:16.678881 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678805 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-kubelet\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.678881 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.678826 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-run-ovn\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.679345 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.679322 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 16 18:40:16.680775 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.680757 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.681353 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.681336 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 16 18:40:16.681526 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.681512 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 16 18:40:16.681695 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.681678 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-q4qr9\"" Sep 16 18:40:16.681982 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.681958 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 16 18:40:16.682650 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.682630 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 16 18:40:16.683472 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.683456 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.684941 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.684921 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4vt5b\"" Sep 16 18:40:16.685141 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.685123 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 16 18:40:16.686719 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.685987 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 16 18:40:16.686719 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.686078 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6f6rf\"" Sep 16 18:40:16.686719 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.686278 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 16 18:40:16.686719 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.686484 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vxc8t" Sep 16 18:40:16.688527 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.688460 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c8dx8\"" Sep 16 18:40:16.688731 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.688708 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:16.688797 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.688760 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 16 18:40:16.688797 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:16.688781 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:16.689052 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.689033 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 16 18:40:16.732060 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.731890 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rkzlm" Sep 16 18:40:16.749012 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.748985 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rkzlm" Sep 16 18:40:16.766517 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.766494 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 16 18:40:16.779029 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779005 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-systemd-units\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.779185 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779042 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/61d20eff-c5cd-43af-aa68-7ee018467a60-ovnkube-config\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.779185 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779068 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-multus-cni-dir\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.779185 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779094 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-etc-kubernetes\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.779185 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779141 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58075b65-6a9b-4ef3-9776-2cf7c3a5e15a-serviceca\") pod \"node-ca-tjc8j\" (UID: \"58075b65-6a9b-4ef3-9776-2cf7c3a5e15a\") " pod="openshift-image-registry/node-ca-tjc8j" Sep 16 18:40:16.779185 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779171 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-run\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.779486 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779197 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-etc-selinux\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.779486 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779223 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-run-ovn-kubernetes\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.779486 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779260 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-run-netns\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.779486 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779313 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0368d478-39bb-4164-8c20-bb8848762359-cni-binary-copy\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.779486 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779343 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w678\" (UniqueName: \"kubernetes.io/projected/0368d478-39bb-4164-8c20-bb8848762359-kube-api-access-2w678\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.779486 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779369 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-run-netns\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.779486 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779398 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-etc-openvswitch\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.779486 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779423 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lws2c\" (UniqueName: \"kubernetes.io/projected/61d20eff-c5cd-43af-aa68-7ee018467a60-kube-api-access-lws2c\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.779486 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779450 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkqxs\" (UniqueName: \"kubernetes.io/projected/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-kube-api-access-gkqxs\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779490 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thx9v\" (UniqueName: \"kubernetes.io/projected/58075b65-6a9b-4ef3-9776-2cf7c3a5e15a-kube-api-access-thx9v\") pod \"node-ca-tjc8j\" (UID: \"58075b65-6a9b-4ef3-9776-2cf7c3a5e15a\") " pod="openshift-image-registry/node-ca-tjc8j" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779510 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-lib-modules\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779526 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779541 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36666946-1b37-4585-bb1f-f8e4461ef3b4-iptables-alerter-script\") pod \"iptables-alerter-m9wjt\" (UID: \"36666946-1b37-4585-bb1f-f8e4461ef3b4\") " pod="openshift-network-operator/iptables-alerter-m9wjt" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779557 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-cni-bin\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779571 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-registration-dir\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779600 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-sys-fs\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779662 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-etc-openvswitch\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779674 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-multus-cni-dir\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779713 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0368d478-39bb-4164-8c20-bb8848762359-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779750 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-run-systemd\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779777 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-sysctl-d\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779785 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-systemd-units\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779789 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-etc-kubernetes\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779802 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-host\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779820 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/61d20eff-c5cd-43af-aa68-7ee018467a60-ovnkube-config\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.780089 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/28b45ecf-82c1-4811-9f2b-e9cdf59a7816-agent-certs\") pod \"konnectivity-agent-vxc8t\" (UID: \"28b45ecf-82c1-4811-9f2b-e9cdf59a7816\") " pod="kube-system/konnectivity-agent-vxc8t" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779834 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-run-netns\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779866 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0368d478-39bb-4164-8c20-bb8848762359-system-cni-dir\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779892 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-run-systemd\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779752 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/58075b65-6a9b-4ef3-9776-2cf7c3a5e15a-serviceca\") pod \"node-ca-tjc8j\" (UID: \"58075b65-6a9b-4ef3-9776-2cf7c3a5e15a\") " pod="openshift-image-registry/node-ca-tjc8j" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779751 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-run-ovn-kubernetes\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/61d20eff-c5cd-43af-aa68-7ee018467a60-env-overrides\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779937 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-run-netns\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779963 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-cni-bin\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.779974 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-socket-dir\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780005 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-sys\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780040 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68\") pod \"network-check-target-s4dw9\" (UID: \"3177af74-fcce-4973-9f87-3dea0506ae3d\") " pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780070 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-var-lib-cni-bin\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780108 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-multus-conf-dir\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780114 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-var-lib-cni-bin\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780145 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd09345e-605b-4196-a5d3-1aa3e3010635-tmp-dir\") pod \"node-resolver-ldzq8\" (UID: \"bd09345e-605b-4196-a5d3-1aa3e3010635\") " pod="openshift-dns/node-resolver-ldzq8" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780162 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-multus-conf-dir\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.780918 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780198 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-kubelet-dir\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780224 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-device-dir\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780266 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-sysconfig\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780299 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-var-lib-kubelet\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780339 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0368d478-39bb-4164-8c20-bb8848762359-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780385 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-sysctl-conf\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780410 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/28b45ecf-82c1-4811-9f2b-e9cdf59a7816-konnectivity-ca\") pod \"konnectivity-agent-vxc8t\" (UID: \"28b45ecf-82c1-4811-9f2b-e9cdf59a7816\") " pod="kube-system/konnectivity-agent-vxc8t" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780443 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/61d20eff-c5cd-43af-aa68-7ee018467a60-ovn-node-metrics-cert\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780470 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/61d20eff-c5cd-43af-aa68-7ee018467a60-ovnkube-script-lib\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780497 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-var-lib-cni-multus\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780506 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd09345e-605b-4196-a5d3-1aa3e3010635-tmp-dir\") pod \"node-resolver-ldzq8\" (UID: \"bd09345e-605b-4196-a5d3-1aa3e3010635\") " pod="openshift-dns/node-resolver-ldzq8" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780605 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-var-lib-cni-multus\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780632 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-var-lib-kubelet\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-multus-daemon-config\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780689 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-run-multus-certs\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780732 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-var-lib-kubelet\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780744 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-run-multus-certs\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.781813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780771 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw8mn\" (UniqueName: \"kubernetes.io/projected/bd09345e-605b-4196-a5d3-1aa3e3010635-kube-api-access-nw8mn\") pod \"node-resolver-ldzq8\" (UID: \"bd09345e-605b-4196-a5d3-1aa3e3010635\") " pod="openshift-dns/node-resolver-ldzq8" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780800 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58075b65-6a9b-4ef3-9776-2cf7c3a5e15a-host\") pod \"node-ca-tjc8j\" (UID: \"58075b65-6a9b-4ef3-9776-2cf7c3a5e15a\") " pod="openshift-image-registry/node-ca-tjc8j" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-run-openvswitch\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780856 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v8rj\" (UniqueName: \"kubernetes.io/projected/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-kube-api-access-4v8rj\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780903 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/58075b65-6a9b-4ef3-9776-2cf7c3a5e15a-host\") pod \"node-ca-tjc8j\" (UID: \"58075b65-6a9b-4ef3-9776-2cf7c3a5e15a\") " pod="openshift-image-registry/node-ca-tjc8j" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780929 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-systemd\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780950 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-run-openvswitch\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780955 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-tuned\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780983 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36666946-1b37-4585-bb1f-f8e4461ef3b4-host-slash\") pod \"iptables-alerter-m9wjt\" (UID: \"36666946-1b37-4585-bb1f-f8e4461ef3b4\") " pod="openshift-network-operator/iptables-alerter-m9wjt" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.780989 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781019 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-slash\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781053 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-slash\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781052 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/61d20eff-c5cd-43af-aa68-7ee018467a60-ovnkube-script-lib\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-var-lib-openvswitch\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781087 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/61d20eff-c5cd-43af-aa68-7ee018467a60-env-overrides\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781106 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-system-cni-dir\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781129 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781154 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-modprobe-d\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.782790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781169 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-var-lib-openvswitch\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781179 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drr7b\" (UniqueName: \"kubernetes.io/projected/b83a34db-f76e-4f19-b377-32bba0f5fa2b-kube-api-access-drr7b\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781234 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0368d478-39bb-4164-8c20-bb8848762359-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781257 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-system-cni-dir\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:16.781307 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781360 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781394 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-hostroot\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781422 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-kubernetes\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781489 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6jfr\" (UniqueName: \"kubernetes.io/projected/36666946-1b37-4585-bb1f-f8e4461ef3b4-kube-api-access-h6jfr\") pod \"iptables-alerter-m9wjt\" (UID: \"36666946-1b37-4585-bb1f-f8e4461ef3b4\") " pod="openshift-network-operator/iptables-alerter-m9wjt" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781520 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-run-k8s-cni-cncf-io\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781547 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/664c47f5-9768-4395-8396-ce9ac30421e3-kubelet-config\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781574 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxgt7\" (UniqueName: \"kubernetes.io/projected/cf61077c-6034-4351-a341-f05f0e4a8b8f-kube-api-access-cxgt7\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781589 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-multus-daemon-config\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:16.781656 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret podName:664c47f5-9768-4395-8396-ce9ac30421e3 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:17.28163577 +0000 UTC m=+2.962490402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret") pod "global-pull-secret-syncer-swmrz" (UID: "664c47f5-9768-4395-8396-ce9ac30421e3") : object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781655 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-host-run-k8s-cni-cncf-io\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781695 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.783780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781739 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-hostroot\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781793 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/664c47f5-9768-4395-8396-ce9ac30421e3-kubelet-config\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781826 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-cni-netd\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781851 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-cni-netd\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781855 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-cni-binary-copy\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781906 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0368d478-39bb-4164-8c20-bb8848762359-cnibin\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781946 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-node-log\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.781976 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-log-socket\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-cnibin\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782025 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/664c47f5-9768-4395-8396-ce9ac30421e3-dbus\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782051 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf61077c-6034-4351-a341-f05f0e4a8b8f-tmp\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782063 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-log-socket\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782074 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-kubelet\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782108 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-run-ovn\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782112 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-host-kubelet\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782137 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-multus-socket-dir-parent\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782164 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-cnibin\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782167 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd09345e-605b-4196-a5d3-1aa3e3010635-hosts-file\") pod \"node-resolver-ldzq8\" (UID: \"bd09345e-605b-4196-a5d3-1aa3e3010635\") " pod="openshift-dns/node-resolver-ldzq8" Sep 16 18:40:16.784795 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782260 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/664c47f5-9768-4395-8396-ce9ac30421e3-dbus\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:16.787444 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782282 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bd09345e-605b-4196-a5d3-1aa3e3010635-hosts-file\") pod \"node-resolver-ldzq8\" (UID: \"bd09345e-605b-4196-a5d3-1aa3e3010635\") " pod="openshift-dns/node-resolver-ldzq8" Sep 16 18:40:16.787444 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782310 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-run-ovn\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.787444 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782339 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-multus-socket-dir-parent\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.787444 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782341 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-cni-binary-copy\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.787444 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.782359 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/61d20eff-c5cd-43af-aa68-7ee018467a60-node-log\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.787444 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.785546 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/61d20eff-c5cd-43af-aa68-7ee018467a60-ovn-node-metrics-cert\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.792335 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.792309 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lws2c\" (UniqueName: \"kubernetes.io/projected/61d20eff-c5cd-43af-aa68-7ee018467a60-kube-api-access-lws2c\") pod \"ovnkube-node-b55tw\" (UID: \"61d20eff-c5cd-43af-aa68-7ee018467a60\") " pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.797923 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.797877 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-99.ec2.internal" event={"ID":"d5acb9fe33c38810a5100cf8c524f161","Type":"ContainerStarted","Data":"0c83f3418cc2b0905dfc48ead980bb0e0db132f416325c77ef19d07befd1361d"} Sep 16 18:40:16.801095 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.801020 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" event={"ID":"53c18eb6ba51406b052ec7b9ec1cf464","Type":"ContainerStarted","Data":"066a9f7473250987a25888b2b93bc6c5865f8eece9b88eab5077a28d635f4991"} Sep 16 18:40:16.802187 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.802164 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thx9v\" (UniqueName: \"kubernetes.io/projected/58075b65-6a9b-4ef3-9776-2cf7c3a5e15a-kube-api-access-thx9v\") pod \"node-ca-tjc8j\" (UID: \"58075b65-6a9b-4ef3-9776-2cf7c3a5e15a\") " pod="openshift-image-registry/node-ca-tjc8j" Sep 16 18:40:16.802315 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.802295 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw8mn\" (UniqueName: \"kubernetes.io/projected/bd09345e-605b-4196-a5d3-1aa3e3010635-kube-api-access-nw8mn\") pod \"node-resolver-ldzq8\" (UID: \"bd09345e-605b-4196-a5d3-1aa3e3010635\") " pod="openshift-dns/node-resolver-ldzq8" Sep 16 18:40:16.802446 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.802422 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkqxs\" (UniqueName: \"kubernetes.io/projected/a0e134d9-fa78-4ee8-8ca2-489a1e626de0-kube-api-access-gkqxs\") pod \"multus-vkgn2\" (UID: \"a0e134d9-fa78-4ee8-8ca2-489a1e626de0\") " pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.882557 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882482 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v8rj\" (UniqueName: \"kubernetes.io/projected/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-kube-api-access-4v8rj\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.882557 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882529 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-systemd\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.882557 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882557 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-tuned\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882582 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36666946-1b37-4585-bb1f-f8e4461ef3b4-host-slash\") pod \"iptables-alerter-m9wjt\" (UID: \"36666946-1b37-4585-bb1f-f8e4461ef3b4\") " pod="openshift-network-operator/iptables-alerter-m9wjt" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882623 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-modprobe-d\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882650 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drr7b\" (UniqueName: \"kubernetes.io/projected/b83a34db-f76e-4f19-b377-32bba0f5fa2b-kube-api-access-drr7b\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882678 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0368d478-39bb-4164-8c20-bb8848762359-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882679 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-systemd\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882704 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-kubernetes\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882709 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36666946-1b37-4585-bb1f-f8e4461ef3b4-host-slash\") pod \"iptables-alerter-m9wjt\" (UID: \"36666946-1b37-4585-bb1f-f8e4461ef3b4\") " pod="openshift-network-operator/iptables-alerter-m9wjt" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882765 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-kubernetes\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882833 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6jfr\" (UniqueName: \"kubernetes.io/projected/36666946-1b37-4585-bb1f-f8e4461ef3b4-kube-api-access-h6jfr\") pod \"iptables-alerter-m9wjt\" (UID: \"36666946-1b37-4585-bb1f-f8e4461ef3b4\") " pod="openshift-network-operator/iptables-alerter-m9wjt" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882847 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-modprobe-d\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882878 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxgt7\" (UniqueName: \"kubernetes.io/projected/cf61077c-6034-4351-a341-f05f0e4a8b8f-kube-api-access-cxgt7\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0368d478-39bb-4164-8c20-bb8848762359-cnibin\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.882922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882929 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0368d478-39bb-4164-8c20-bb8848762359-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882941 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf61077c-6034-4351-a341-f05f0e4a8b8f-tmp\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882974 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-run\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882982 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0368d478-39bb-4164-8c20-bb8848762359-cnibin\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.882998 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-etc-selinux\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883066 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-etc-selinux\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883066 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-run\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883118 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0368d478-39bb-4164-8c20-bb8848762359-cni-binary-copy\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w678\" (UniqueName: \"kubernetes.io/projected/0368d478-39bb-4164-8c20-bb8848762359-kube-api-access-2w678\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883186 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-lib-modules\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883235 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883282 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36666946-1b37-4585-bb1f-f8e4461ef3b4-iptables-alerter-script\") pod \"iptables-alerter-m9wjt\" (UID: \"36666946-1b37-4585-bb1f-f8e4461ef3b4\") " pod="openshift-network-operator/iptables-alerter-m9wjt" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883317 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-lib-modules\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883337 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-registration-dir\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883364 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-sys-fs\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883392 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0368d478-39bb-4164-8c20-bb8848762359-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:16.883408 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:16.883595 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883430 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-sysctl-d\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883454 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-host\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:16.883478 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs podName:b83a34db-f76e-4f19-b377-32bba0f5fa2b nodeName:}" failed. No retries permitted until 2026-09-16 18:40:17.383457333 +0000 UTC m=+3.064311951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs") pod "network-metrics-daemon-c2scp" (UID: "b83a34db-f76e-4f19-b377-32bba0f5fa2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/28b45ecf-82c1-4811-9f2b-e9cdf59a7816-agent-certs\") pod \"konnectivity-agent-vxc8t\" (UID: \"28b45ecf-82c1-4811-9f2b-e9cdf59a7816\") " pod="kube-system/konnectivity-agent-vxc8t" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883519 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-host\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883536 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0368d478-39bb-4164-8c20-bb8848762359-system-cni-dir\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883563 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-socket-dir\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883587 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-sys\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68\") pod \"network-check-target-s4dw9\" (UID: \"3177af74-fcce-4973-9f87-3dea0506ae3d\") " pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883641 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-kubelet-dir\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-device-dir\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-sysconfig\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883715 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-var-lib-kubelet\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883730 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-registration-dir\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883741 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0368d478-39bb-4164-8c20-bb8848762359-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883767 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-sysctl-conf\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.884477 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883792 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/28b45ecf-82c1-4811-9f2b-e9cdf59a7816-konnectivity-ca\") pod \"konnectivity-agent-vxc8t\" (UID: \"28b45ecf-82c1-4811-9f2b-e9cdf59a7816\") " pod="kube-system/konnectivity-agent-vxc8t" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883811 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-socket-dir\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883908 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0368d478-39bb-4164-8c20-bb8848762359-system-cni-dir\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883970 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-sys-fs\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883992 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-sysctl-d\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.883998 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0368d478-39bb-4164-8c20-bb8848762359-cni-binary-copy\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.884021 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-sys\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.884082 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-sysconfig\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.884136 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-kubelet-dir\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.884180 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/36666946-1b37-4585-bb1f-f8e4461ef3b4-iptables-alerter-script\") pod \"iptables-alerter-m9wjt\" (UID: \"36666946-1b37-4585-bb1f-f8e4461ef3b4\") " pod="openshift-network-operator/iptables-alerter-m9wjt" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.884240 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0368d478-39bb-4164-8c20-bb8848762359-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.884279 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-device-dir\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.884306 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-var-lib-kubelet\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.884423 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-sysctl-conf\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.884436 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/28b45ecf-82c1-4811-9f2b-e9cdf59a7816-konnectivity-ca\") pod \"konnectivity-agent-vxc8t\" (UID: \"28b45ecf-82c1-4811-9f2b-e9cdf59a7816\") " pod="kube-system/konnectivity-agent-vxc8t" Sep 16 18:40:16.885213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.884611 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0368d478-39bb-4164-8c20-bb8848762359-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.885978 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.885799 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cf61077c-6034-4351-a341-f05f0e4a8b8f-etc-tuned\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.886577 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.886542 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cf61077c-6034-4351-a341-f05f0e4a8b8f-tmp\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.887140 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.887116 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/28b45ecf-82c1-4811-9f2b-e9cdf59a7816-agent-certs\") pod \"konnectivity-agent-vxc8t\" (UID: \"28b45ecf-82c1-4811-9f2b-e9cdf59a7816\") " pod="kube-system/konnectivity-agent-vxc8t" Sep 16 18:40:16.896942 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.896881 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v8rj\" (UniqueName: \"kubernetes.io/projected/d36fc0d3-5f9b-4fc2-96a0-10b7df0da777-kube-api-access-4v8rj\") pod \"aws-ebs-csi-driver-node-62wft\" (UID: \"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:16.897170 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.897141 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drr7b\" (UniqueName: \"kubernetes.io/projected/b83a34db-f76e-4f19-b377-32bba0f5fa2b-kube-api-access-drr7b\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:16.931932 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:16.931900 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 18:40:16.931932 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:16.931934 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 18:40:16.931932 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:16.931951 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fpp68 for pod openshift-network-diagnostics/network-check-target-s4dw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:16.932192 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:16.932040 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68 podName:3177af74-fcce-4973-9f87-3dea0506ae3d nodeName:}" failed. No retries permitted until 2026-09-16 18:40:17.432018535 +0000 UTC m=+3.112873173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fpp68" (UniqueName: "kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68") pod "network-check-target-s4dw9" (UID: "3177af74-fcce-4973-9f87-3dea0506ae3d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:16.935412 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.935383 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w678\" (UniqueName: \"kubernetes.io/projected/0368d478-39bb-4164-8c20-bb8848762359-kube-api-access-2w678\") pod \"multus-additional-cni-plugins-8fkdw\" (UID: \"0368d478-39bb-4164-8c20-bb8848762359\") " pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:16.935613 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.935589 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6jfr\" (UniqueName: \"kubernetes.io/projected/36666946-1b37-4585-bb1f-f8e4461ef3b4-kube-api-access-h6jfr\") pod \"iptables-alerter-m9wjt\" (UID: \"36666946-1b37-4585-bb1f-f8e4461ef3b4\") " pod="openshift-network-operator/iptables-alerter-m9wjt" Sep 16 18:40:16.935707 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.935641 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxgt7\" (UniqueName: \"kubernetes.io/projected/cf61077c-6034-4351-a341-f05f0e4a8b8f-kube-api-access-cxgt7\") pod \"tuned-fm6nk\" (UID: \"cf61077c-6034-4351-a341-f05f0e4a8b8f\") " pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:16.972503 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.972478 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:16.984828 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.984806 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vkgn2" Sep 16 18:40:16.996847 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:16.996821 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" Sep 16 18:40:17.002774 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:17.002752 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ldzq8" Sep 16 18:40:17.011644 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:17.011625 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tjc8j" Sep 16 18:40:17.020408 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:17.020388 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m9wjt" Sep 16 18:40:17.028309 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:17.028288 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" Sep 16 18:40:17.036179 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:17.036156 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8fkdw" Sep 16 18:40:17.041093 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:17.041068 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vxc8t" Sep 16 18:40:17.286491 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:17.286442 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:17.286700 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:17.286587 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:17.286700 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:17.286663 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret podName:664c47f5-9768-4395-8396-ce9ac30421e3 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:18.286642735 +0000 UTC m=+3.967497351 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret") pod "global-pull-secret-syncer-swmrz" (UID: "664c47f5-9768-4395-8396-ce9ac30421e3") : object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:17.387044 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:17.387004 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:17.387235 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:17.387148 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:17.387235 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:17.387214 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs podName:b83a34db-f76e-4f19-b377-32bba0f5fa2b nodeName:}" failed. No retries permitted until 2026-09-16 18:40:18.38719893 +0000 UTC m=+4.068053541 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs") pod "network-metrics-daemon-c2scp" (UID: "b83a34db-f76e-4f19-b377-32bba0f5fa2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:17.487835 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:17.487798 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68\") pod \"network-check-target-s4dw9\" (UID: \"3177af74-fcce-4973-9f87-3dea0506ae3d\") " pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:17.488019 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:17.487979 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 18:40:17.488019 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:17.488001 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 18:40:17.488019 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:17.488014 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fpp68 for pod openshift-network-diagnostics/network-check-target-s4dw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:17.488182 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:17.488080 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68 podName:3177af74-fcce-4973-9f87-3dea0506ae3d nodeName:}" failed. No retries permitted until 2026-09-16 18:40:18.48806082 +0000 UTC m=+4.168915456 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fpp68" (UniqueName: "kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68") pod "network-check-target-s4dw9" (UID: "3177af74-fcce-4973-9f87-3dea0506ae3d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:17.752691 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:17.752595 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:35:17 +0000 UTC" deadline="2028-02-21 20:59:31.85978493 +0000 UTC" Sep 16 18:40:17.752691 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:17.752640 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12554h19m14.107150724s" Sep 16 18:40:17.791382 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:17.791351 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:17.791516 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:17.791469 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:17.978912 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:17.978886 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36666946_1b37_4585_bb1f_f8e4461ef3b4.slice/crio-0bfc6c79bc8f19cd2137141770152bc2aa9856e3b1de63176453ac100eb129e7 WatchSource:0}: Error finding container 0bfc6c79bc8f19cd2137141770152bc2aa9856e3b1de63176453ac100eb129e7: Status 404 returned error can't find the container with id 0bfc6c79bc8f19cd2137141770152bc2aa9856e3b1de63176453ac100eb129e7 Sep 16 18:40:17.979957 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:17.979929 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf61077c_6034_4351_a341_f05f0e4a8b8f.slice/crio-6c95769f72a0230d5bf96f7cc81a6ea02ebe7b4b456dd529d21f7ca0d240ed62 WatchSource:0}: Error finding container 6c95769f72a0230d5bf96f7cc81a6ea02ebe7b4b456dd529d21f7ca0d240ed62: Status 404 returned error can't find the container with id 6c95769f72a0230d5bf96f7cc81a6ea02ebe7b4b456dd529d21f7ca0d240ed62 Sep 16 18:40:17.981414 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:17.981378 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61d20eff_c5cd_43af_aa68_7ee018467a60.slice/crio-9c83876f5d433ed1fcb7361b7d295a046dbe646e723ccb26aca46877f603ae65 WatchSource:0}: Error finding container 9c83876f5d433ed1fcb7361b7d295a046dbe646e723ccb26aca46877f603ae65: Status 404 returned error can't find the container with id 9c83876f5d433ed1fcb7361b7d295a046dbe646e723ccb26aca46877f603ae65 Sep 16 18:40:17.983590 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:17.983568 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd09345e_605b_4196_a5d3_1aa3e3010635.slice/crio-46a31b4330f5b8dfcaaca640da58f883a49b54af8096295f12ff60d4b630218b WatchSource:0}: Error finding container 46a31b4330f5b8dfcaaca640da58f883a49b54af8096295f12ff60d4b630218b: Status 404 returned error can't find the container with id 46a31b4330f5b8dfcaaca640da58f883a49b54af8096295f12ff60d4b630218b Sep 16 18:40:17.987767 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:17.987746 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36fc0d3_5f9b_4fc2_96a0_10b7df0da777.slice/crio-d9189d95a2a8f41719ccec23aaf0400030eb06da70ef6f12e63a55969eff9679 WatchSource:0}: Error finding container d9189d95a2a8f41719ccec23aaf0400030eb06da70ef6f12e63a55969eff9679: Status 404 returned error can't find the container with id d9189d95a2a8f41719ccec23aaf0400030eb06da70ef6f12e63a55969eff9679 Sep 16 18:40:17.988687 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:17.988662 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e134d9_fa78_4ee8_8ca2_489a1e626de0.slice/crio-4e8ccd4203997247fc25068c3dc0fc428af5d6ce9715f3e08fed434db6fa8214 WatchSource:0}: Error finding container 4e8ccd4203997247fc25068c3dc0fc428af5d6ce9715f3e08fed434db6fa8214: Status 404 returned error can't find the container with id 4e8ccd4203997247fc25068c3dc0fc428af5d6ce9715f3e08fed434db6fa8214 Sep 16 18:40:17.990558 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:17.990526 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0368d478_39bb_4164_8c20_bb8848762359.slice/crio-efa0d804e7a4bc3d16db40ed63a19ba4e1421a8eb15ff7d3a6be02c9376055d8 WatchSource:0}: Error finding container efa0d804e7a4bc3d16db40ed63a19ba4e1421a8eb15ff7d3a6be02c9376055d8: Status 404 returned error can't find the container with id efa0d804e7a4bc3d16db40ed63a19ba4e1421a8eb15ff7d3a6be02c9376055d8 Sep 16 18:40:17.991055 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:17.991032 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58075b65_6a9b_4ef3_9776_2cf7c3a5e15a.slice/crio-f8b42afe394791d91eca50e62a052f259d681ec48de6a1dcca7ba1ea0fd37347 WatchSource:0}: Error finding container f8b42afe394791d91eca50e62a052f259d681ec48de6a1dcca7ba1ea0fd37347: Status 404 returned error can't find the container with id f8b42afe394791d91eca50e62a052f259d681ec48de6a1dcca7ba1ea0fd37347 Sep 16 18:40:17.996732 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:17.996708 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b45ecf_82c1_4811_9f2b_e9cdf59a7816.slice/crio-79e0992bac150903d46a57f3a5dc6d997439db9c8e6f1196dd919ddac7ea69df WatchSource:0}: Error finding container 79e0992bac150903d46a57f3a5dc6d997439db9c8e6f1196dd919ddac7ea69df: Status 404 returned error can't find the container with id 79e0992bac150903d46a57f3a5dc6d997439db9c8e6f1196dd919ddac7ea69df Sep 16 18:40:18.294842 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.294689 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:18.294947 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:18.294862 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:18.294947 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:18.294938 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret podName:664c47f5-9768-4395-8396-ce9ac30421e3 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:20.294916588 +0000 UTC m=+5.975771216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret") pod "global-pull-secret-syncer-swmrz" (UID: "664c47f5-9768-4395-8396-ce9ac30421e3") : object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:18.395281 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.395231 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:18.395422 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:18.395385 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:18.395484 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:18.395452 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs podName:b83a34db-f76e-4f19-b377-32bba0f5fa2b nodeName:}" failed. No retries permitted until 2026-09-16 18:40:20.395437364 +0000 UTC m=+6.076291993 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs") pod "network-metrics-daemon-c2scp" (UID: "b83a34db-f76e-4f19-b377-32bba0f5fa2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:18.497169 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.496504 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68\") pod \"network-check-target-s4dw9\" (UID: \"3177af74-fcce-4973-9f87-3dea0506ae3d\") " pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:18.497169 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:18.496717 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 18:40:18.497169 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:18.496736 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 18:40:18.497169 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:18.496750 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fpp68 for pod openshift-network-diagnostics/network-check-target-s4dw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:18.497169 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:18.496809 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68 podName:3177af74-fcce-4973-9f87-3dea0506ae3d nodeName:}" failed. No retries permitted until 2026-09-16 18:40:20.496790542 +0000 UTC m=+6.177645180 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fpp68" (UniqueName: "kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68") pod "network-check-target-s4dw9" (UID: "3177af74-fcce-4973-9f87-3dea0506ae3d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:18.753344 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.753229 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 18:35:17 +0000 UTC" deadline="2028-02-14 21:43:05.204735318 +0000 UTC" Sep 16 18:40:18.753344 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.753292 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12387h2m46.451447895s" Sep 16 18:40:18.793627 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.793599 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:18.793782 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:18.793726 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:18.794155 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.794133 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:18.794275 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:18.794239 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:18.817813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.816774 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m9wjt" event={"ID":"36666946-1b37-4585-bb1f-f8e4461ef3b4","Type":"ContainerStarted","Data":"0bfc6c79bc8f19cd2137141770152bc2aa9856e3b1de63176453ac100eb129e7"} Sep 16 18:40:18.837230 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.837197 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-99.ec2.internal" event={"ID":"d5acb9fe33c38810a5100cf8c524f161","Type":"ContainerStarted","Data":"9752fcbbce66040dd5caf179e241d58499bf2de4837d7ac0f8abe712eb7e655e"} Sep 16 18:40:18.874714 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.874680 2566 generic.go:356] "Generic (PLEG): container finished" podID="53c18eb6ba51406b052ec7b9ec1cf464" containerID="30e3957f0044ac649cee20c74eb10ab5bdd94a5487256f8660f0d4f7beb62d80" exitCode=0 Sep 16 18:40:18.874851 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.874791 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" event={"ID":"53c18eb6ba51406b052ec7b9ec1cf464","Type":"ContainerDied","Data":"30e3957f0044ac649cee20c74eb10ab5bdd94a5487256f8660f0d4f7beb62d80"} Sep 16 18:40:18.882679 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.882650 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vkgn2" event={"ID":"a0e134d9-fa78-4ee8-8ca2-489a1e626de0","Type":"ContainerStarted","Data":"4e8ccd4203997247fc25068c3dc0fc428af5d6ce9715f3e08fed434db6fa8214"} Sep 16 18:40:18.908743 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.908709 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vxc8t" event={"ID":"28b45ecf-82c1-4811-9f2b-e9cdf59a7816","Type":"ContainerStarted","Data":"79e0992bac150903d46a57f3a5dc6d997439db9c8e6f1196dd919ddac7ea69df"} Sep 16 18:40:18.911067 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.911003 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-99.ec2.internal" podStartSLOduration=2.910984354 podStartE2EDuration="2.910984354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:18.861997536 +0000 UTC m=+4.542852172" watchObservedRunningTime="2026-09-16 18:40:18.910984354 +0000 UTC m=+4.591839012" Sep 16 18:40:18.924391 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.924313 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tjc8j" event={"ID":"58075b65-6a9b-4ef3-9776-2cf7c3a5e15a","Type":"ContainerStarted","Data":"f8b42afe394791d91eca50e62a052f259d681ec48de6a1dcca7ba1ea0fd37347"} Sep 16 18:40:18.935345 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.935300 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8fkdw" event={"ID":"0368d478-39bb-4164-8c20-bb8848762359","Type":"ContainerStarted","Data":"efa0d804e7a4bc3d16db40ed63a19ba4e1421a8eb15ff7d3a6be02c9376055d8"} Sep 16 18:40:18.939953 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.939842 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" event={"ID":"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777","Type":"ContainerStarted","Data":"d9189d95a2a8f41719ccec23aaf0400030eb06da70ef6f12e63a55969eff9679"} Sep 16 18:40:18.952955 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.952862 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ldzq8" event={"ID":"bd09345e-605b-4196-a5d3-1aa3e3010635","Type":"ContainerStarted","Data":"46a31b4330f5b8dfcaaca640da58f883a49b54af8096295f12ff60d4b630218b"} Sep 16 18:40:18.983353 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.983321 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" event={"ID":"61d20eff-c5cd-43af-aa68-7ee018467a60","Type":"ContainerStarted","Data":"9c83876f5d433ed1fcb7361b7d295a046dbe646e723ccb26aca46877f603ae65"} Sep 16 18:40:18.996907 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:18.996079 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" event={"ID":"cf61077c-6034-4351-a341-f05f0e4a8b8f","Type":"ContainerStarted","Data":"6c95769f72a0230d5bf96f7cc81a6ea02ebe7b4b456dd529d21f7ca0d240ed62"} Sep 16 18:40:19.791504 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:19.791468 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:19.792090 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:19.791611 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:20.023351 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:20.023316 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" event={"ID":"53c18eb6ba51406b052ec7b9ec1cf464","Type":"ContainerStarted","Data":"f7c6bb833cc55853853471f57f18cc2bce7dcca2773eb7e420d3f56606fa7fed"} Sep 16 18:40:20.311750 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:20.311664 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:20.311918 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:20.311822 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:20.311918 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:20.311903 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret podName:664c47f5-9768-4395-8396-ce9ac30421e3 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:24.311881892 +0000 UTC m=+9.992736519 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret") pod "global-pull-secret-syncer-swmrz" (UID: "664c47f5-9768-4395-8396-ce9ac30421e3") : object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:20.412464 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:20.412422 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:20.412638 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:20.412593 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:20.412709 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:20.412663 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs podName:b83a34db-f76e-4f19-b377-32bba0f5fa2b nodeName:}" failed. No retries permitted until 2026-09-16 18:40:24.412641462 +0000 UTC m=+10.093496090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs") pod "network-metrics-daemon-c2scp" (UID: "b83a34db-f76e-4f19-b377-32bba0f5fa2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:20.513846 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:20.513053 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68\") pod \"network-check-target-s4dw9\" (UID: \"3177af74-fcce-4973-9f87-3dea0506ae3d\") " pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:20.513846 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:20.513315 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 18:40:20.513846 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:20.513343 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 18:40:20.513846 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:20.513357 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fpp68 for pod openshift-network-diagnostics/network-check-target-s4dw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:20.513846 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:20.513423 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68 podName:3177af74-fcce-4973-9f87-3dea0506ae3d nodeName:}" failed. No retries permitted until 2026-09-16 18:40:24.513403828 +0000 UTC m=+10.194258444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fpp68" (UniqueName: "kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68") pod "network-check-target-s4dw9" (UID: "3177af74-fcce-4973-9f87-3dea0506ae3d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:20.792944 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:20.792429 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:20.792944 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:20.792561 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:20.793797 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:20.793615 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:20.793797 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:20.793725 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:21.791290 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:21.791251 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:21.791494 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:21.791419 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:22.792314 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:22.792287 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:22.792760 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:22.792414 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:22.794302 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:22.794273 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:22.794417 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:22.794397 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:23.791033 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:23.790997 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:23.791290 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:23.791125 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:24.346657 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:24.346467 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:24.346657 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:24.346643 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:24.347333 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:24.346703 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret podName:664c47f5-9768-4395-8396-ce9ac30421e3 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:32.346685896 +0000 UTC m=+18.027540505 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret") pod "global-pull-secret-syncer-swmrz" (UID: "664c47f5-9768-4395-8396-ce9ac30421e3") : object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:24.448060 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:24.447569 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:24.448060 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:24.447687 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:24.448060 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:24.447759 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs podName:b83a34db-f76e-4f19-b377-32bba0f5fa2b nodeName:}" failed. No retries permitted until 2026-09-16 18:40:32.447741001 +0000 UTC m=+18.128595626 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs") pod "network-metrics-daemon-c2scp" (UID: "b83a34db-f76e-4f19-b377-32bba0f5fa2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:24.547980 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:24.547944 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68\") pod \"network-check-target-s4dw9\" (UID: \"3177af74-fcce-4973-9f87-3dea0506ae3d\") " pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:24.548150 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:24.548097 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 18:40:24.548150 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:24.548122 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 18:40:24.548150 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:24.548134 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fpp68 for pod openshift-network-diagnostics/network-check-target-s4dw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:24.548305 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:24.548207 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68 podName:3177af74-fcce-4973-9f87-3dea0506ae3d nodeName:}" failed. No retries permitted until 2026-09-16 18:40:32.548173682 +0000 UTC m=+18.229028311 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fpp68" (UniqueName: "kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68") pod "network-check-target-s4dw9" (UID: "3177af74-fcce-4973-9f87-3dea0506ae3d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:24.792003 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:24.791973 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:24.792177 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:24.792074 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:24.792177 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:24.792154 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:24.793279 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:24.793164 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:25.791228 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:25.791173 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:25.791734 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:25.791321 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:26.791668 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:26.791637 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:26.792099 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:26.791778 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:26.792099 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:26.791828 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:26.792099 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:26.791946 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:27.791645 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:27.791605 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:27.791794 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:27.791755 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:28.791051 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:28.791013 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:28.791237 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:28.791013 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:28.791237 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:28.791154 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:28.791237 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:28.791227 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:29.791752 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:29.791717 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:29.792115 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:29.791847 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:30.791848 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:30.791813 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:30.792321 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:30.791929 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:30.792321 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:30.791814 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:30.792452 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:30.792383 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:31.791597 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:31.791565 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:31.791806 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:31.791690 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:32.403296 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:32.403267 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:32.403582 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:32.403359 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:32.403582 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:32.403427 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret podName:664c47f5-9768-4395-8396-ce9ac30421e3 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:48.403405721 +0000 UTC m=+34.084260345 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret") pod "global-pull-secret-syncer-swmrz" (UID: "664c47f5-9768-4395-8396-ce9ac30421e3") : object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:32.503838 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:32.503808 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:32.503938 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:32.503923 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:32.503988 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:32.503967 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs podName:b83a34db-f76e-4f19-b377-32bba0f5fa2b nodeName:}" failed. No retries permitted until 2026-09-16 18:40:48.503955092 +0000 UTC m=+34.184809702 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs") pod "network-metrics-daemon-c2scp" (UID: "b83a34db-f76e-4f19-b377-32bba0f5fa2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:32.604775 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:32.604701 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68\") pod \"network-check-target-s4dw9\" (UID: \"3177af74-fcce-4973-9f87-3dea0506ae3d\") " pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:32.604890 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:32.604866 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 18:40:32.604890 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:32.604889 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 18:40:32.604952 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:32.604898 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fpp68 for pod openshift-network-diagnostics/network-check-target-s4dw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:32.604952 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:32.604944 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68 podName:3177af74-fcce-4973-9f87-3dea0506ae3d nodeName:}" failed. No retries permitted until 2026-09-16 18:40:48.604929608 +0000 UTC m=+34.285784218 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fpp68" (UniqueName: "kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68") pod "network-check-target-s4dw9" (UID: "3177af74-fcce-4973-9f87-3dea0506ae3d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:32.791622 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:32.791591 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:32.791780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:32.791589 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:32.791780 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:32.791709 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:32.791894 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:32.791866 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:33.790977 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:33.790946 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:33.791454 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:33.791049 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:34.792492 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:34.792459 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:34.792930 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:34.792568 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:34.792930 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:34.792604 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:34.792930 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:34.792669 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:35.790851 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:35.790825 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:35.791004 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:35.790930 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:36.058972 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:36.058489 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vxc8t" event={"ID":"28b45ecf-82c1-4811-9f2b-e9cdf59a7816","Type":"ContainerStarted","Data":"f4c180c1077b6fe84eb412f6e34a026ac17900a1f4568520f5a9424f64e46ba4"} Sep 16 18:40:36.061482 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:36.061457 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" event={"ID":"61d20eff-c5cd-43af-aa68-7ee018467a60","Type":"ContainerStarted","Data":"fe1e04bbc79849160ab8d1150eccde9b430f5f6fc4005c9646bfb40b694b37f8"} Sep 16 18:40:36.067515 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:36.067487 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" event={"ID":"cf61077c-6034-4351-a341-f05f0e4a8b8f","Type":"ContainerStarted","Data":"a01c880bf0592923cf98397e0fbf908ba612f85b316ac5dd6ca3ba24c13a1193"} Sep 16 18:40:36.078330 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:36.078281 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-99.ec2.internal" podStartSLOduration=20.07826402 podStartE2EDuration="20.07826402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:20.044156018 +0000 UTC m=+5.725010654" watchObservedRunningTime="2026-09-16 18:40:36.07826402 +0000 UTC m=+21.759118645" Sep 16 18:40:36.099967 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:36.099873 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vxc8t" podStartSLOduration=6.79129376 podStartE2EDuration="21.099852001s" totalImagesPullingTime="14.308558241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:36.078397921 +0000 UTC m=+21.759252553" watchObservedRunningTime="2026-09-16 18:40:36.099852001 +0000 UTC m=+21.780706635" Sep 16 18:40:36.100046 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:36.099985 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fm6nk" podStartSLOduration=3.203253126 podStartE2EDuration="21.099978249s" totalImagesPullingTime="17.896725123s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:36.099664095 +0000 UTC m=+21.780518726" watchObservedRunningTime="2026-09-16 18:40:36.099978249 +0000 UTC m=+21.780832882" Sep 16 18:40:36.791973 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:36.791689 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:36.791973 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:36.791689 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:36.792199 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:36.792016 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:36.792427 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:36.792398 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:37.070931 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.070904 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tjc8j" event={"ID":"58075b65-6a9b-4ef3-9776-2cf7c3a5e15a","Type":"ContainerStarted","Data":"34db7a73a83ba2df448b9ae78888f2bdfde6584cb71208ed389c491fa6b2c830"} Sep 16 18:40:37.072886 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.072861 2566 generic.go:356] "Generic (PLEG): container finished" podID="0368d478-39bb-4164-8c20-bb8848762359" containerID="ecd8dcec1df367b1e47abcb1da09ebb55ed74b4367bc84f9856519f81152121b" exitCode=0 Sep 16 18:40:37.072990 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.072930 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8fkdw" event={"ID":"0368d478-39bb-4164-8c20-bb8848762359","Type":"ContainerDied","Data":"ecd8dcec1df367b1e47abcb1da09ebb55ed74b4367bc84f9856519f81152121b"} Sep 16 18:40:37.074462 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.074438 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" event={"ID":"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777","Type":"ContainerStarted","Data":"3e7146a5cdec19a97097dd8200f319620266666a0748f15d12061a0ebd83591b"} Sep 16 18:40:37.075776 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.075754 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ldzq8" event={"ID":"bd09345e-605b-4196-a5d3-1aa3e3010635","Type":"ContainerStarted","Data":"e00163f41035df6f400e9a904dbf3330047269a9e61ecfd58c164eefe0054b39"} Sep 16 18:40:37.078479 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.078458 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" event={"ID":"61d20eff-c5cd-43af-aa68-7ee018467a60","Type":"ContainerStarted","Data":"89e39cb3ffb4ec2b15ece8a25a5d22115fe88bff17dd787ecdd6c72818c12649"} Sep 16 18:40:37.078573 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.078516 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" event={"ID":"61d20eff-c5cd-43af-aa68-7ee018467a60","Type":"ContainerStarted","Data":"3f5494c3ba841b4fda8456ae33c66c79dddc6a3bf8e21511a883d28d9933e786"} Sep 16 18:40:37.078573 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.078553 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" event={"ID":"61d20eff-c5cd-43af-aa68-7ee018467a60","Type":"ContainerStarted","Data":"f6acc6e87711af269575f7d59a1daa867d38f7b4a0218ed9a18a511649e808a3"} Sep 16 18:40:37.078674 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.078589 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" event={"ID":"61d20eff-c5cd-43af-aa68-7ee018467a60","Type":"ContainerStarted","Data":"fa20d71ddc627c29a0b23585f0363263b2ced3e53b55b7d6d78f27d6d34f04ef"} Sep 16 18:40:37.078674 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.078648 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" event={"ID":"61d20eff-c5cd-43af-aa68-7ee018467a60","Type":"ContainerStarted","Data":"42e818851180e8d0411612474228a797a3059da107750f60f6a151680189e3ae"} Sep 16 18:40:37.079796 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.079780 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vkgn2" event={"ID":"a0e134d9-fa78-4ee8-8ca2-489a1e626de0","Type":"ContainerStarted","Data":"b48212041aad09bda0dd6db7b6fc565dc288c9d651a2d5237efb5863d8dba06e"} Sep 16 18:40:37.100983 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.100947 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tjc8j" podStartSLOduration=4.220093208 podStartE2EDuration="22.100934538s" totalImagesPullingTime="17.88084133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:37.100310181 +0000 UTC m=+22.781164813" watchObservedRunningTime="2026-09-16 18:40:37.100934538 +0000 UTC m=+22.781789170" Sep 16 18:40:37.206514 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.206469 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vkgn2" podStartSLOduration=4.199920568 podStartE2EDuration="22.206455247s" totalImagesPullingTime="18.006534679s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:37.177099808 +0000 UTC m=+22.857954440" watchObservedRunningTime="2026-09-16 18:40:37.206455247 +0000 UTC m=+22.887309861" Sep 16 18:40:37.224930 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.224899 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ldzq8" podStartSLOduration=4.333853453 podStartE2EDuration="22.224887845s" totalImagesPullingTime="17.891034392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:37.22456041 +0000 UTC m=+22.905415042" watchObservedRunningTime="2026-09-16 18:40:37.224887845 +0000 UTC m=+22.905742477" Sep 16 18:40:37.387304 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.387159 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 16 18:40:37.517519 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.517496 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vxc8t" Sep 16 18:40:37.517998 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.517981 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vxc8t" Sep 16 18:40:37.724837 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.724703 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T18:40:37.38729987Z","UUID":"5d909e2d-3b1f-4eb4-a842-64572b39de96","Handler":null,"Name":"","Endpoint":""} Sep 16 18:40:37.726234 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.726213 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 16 18:40:37.726344 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.726244 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 16 18:40:37.791414 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:37.791390 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:37.791530 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:37.791510 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:38.084608 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:38.084575 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m9wjt" event={"ID":"36666946-1b37-4585-bb1f-f8e4461ef3b4","Type":"ContainerStarted","Data":"e2bec6e22a8c289c031d6fdf8c69b4d4d0f5a47a14536375d604b90ecffb022f"} Sep 16 18:40:38.087163 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:38.087134 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" event={"ID":"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777","Type":"ContainerStarted","Data":"514e5b5309c7a4eae1f136d96826179c9778708cd8ff44a2b6ee57be968d3664"} Sep 16 18:40:38.087773 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:38.087752 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vxc8t" Sep 16 18:40:38.087941 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:38.087911 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vxc8t" Sep 16 18:40:38.104453 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:38.104041 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m9wjt" podStartSLOduration=5.207895854 podStartE2EDuration="23.104025027s" totalImagesPullingTime="17.896129173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:38.103958105 +0000 UTC m=+23.784812737" watchObservedRunningTime="2026-09-16 18:40:38.104025027 +0000 UTC m=+23.784879661" Sep 16 18:40:38.791374 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:38.791344 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:38.791523 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:38.791469 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:38.791576 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:38.791521 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:38.791654 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:38.791632 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:39.092166 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:39.092089 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" event={"ID":"d36fc0d3-5f9b-4fc2-96a0-10b7df0da777","Type":"ContainerStarted","Data":"c66d046cd4d1c14333c6d2ef57fa06c952d2a86ce2a84f10261c3409bdc16cc8"} Sep 16 18:40:39.099952 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:39.099881 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" event={"ID":"61d20eff-c5cd-43af-aa68-7ee018467a60","Type":"ContainerStarted","Data":"93118d33d3c467a9c7eddb091b8616c314ba45d5f9a03ef9876c12e3e723c600"} Sep 16 18:40:39.116798 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:39.116758 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-62wft" podStartSLOduration=3.665829727 podStartE2EDuration="24.116742158s" totalImagesPullingTime="20.450912431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:39.116156763 +0000 UTC m=+24.797011395" watchObservedRunningTime="2026-09-16 18:40:39.116742158 +0000 UTC m=+24.797596790" Sep 16 18:40:39.791698 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:39.791667 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:39.791863 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:39.791804 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:40.790843 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:40.790809 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:40.791261 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:40.790936 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:40.791261 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:40.790976 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:40.791261 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:40.791051 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:41.791865 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:41.791713 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:41.792303 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:41.791993 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:42.107534 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:42.107463 2566 generic.go:356] "Generic (PLEG): container finished" podID="0368d478-39bb-4164-8c20-bb8848762359" containerID="3a0a969a766eec41813bd79c0d5072f39ae940a55cfc803347058ae31ab240ed" exitCode=0 Sep 16 18:40:42.107534 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:42.107524 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8fkdw" event={"ID":"0368d478-39bb-4164-8c20-bb8848762359","Type":"ContainerDied","Data":"3a0a969a766eec41813bd79c0d5072f39ae940a55cfc803347058ae31ab240ed"} Sep 16 18:40:42.111434 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:42.111406 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" event={"ID":"61d20eff-c5cd-43af-aa68-7ee018467a60","Type":"ContainerStarted","Data":"615fee38e2f9161401f2970571fe66ff92c46c45db48fa1b28963485594272f5"} Sep 16 18:40:42.111648 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:42.111634 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:42.111708 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:42.111676 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:42.126000 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:42.125982 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:42.180028 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:42.178866 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" podStartSLOduration=9.24343424 podStartE2EDuration="27.178849522s" totalImagesPullingTime="17.935415282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:42.178012088 +0000 UTC m=+27.858866721" watchObservedRunningTime="2026-09-16 18:40:42.178849522 +0000 UTC m=+27.859704156" Sep 16 18:40:42.791709 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:42.791670 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:42.791709 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:42.791710 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:42.792572 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:42.791821 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:42.792572 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:42.791960 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:43.113580 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:43.113521 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:43.126842 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:43.126820 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:40:43.335910 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:43.335885 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-swmrz"] Sep 16 18:40:43.336024 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:43.335987 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:43.336119 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:43.336096 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:43.347671 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:43.347646 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c2scp"] Sep 16 18:40:43.347778 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:43.347743 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:43.347850 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:43.347830 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:43.358178 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:43.358157 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s4dw9"] Sep 16 18:40:43.358277 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:43.358266 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:43.358364 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:43.358348 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:44.117503 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:44.117471 2566 generic.go:356] "Generic (PLEG): container finished" podID="0368d478-39bb-4164-8c20-bb8848762359" containerID="d55edcc3451a21be5bf2ed3677878e8851e6b869db2604df781a087e5bbaec0e" exitCode=0 Sep 16 18:40:44.118249 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:44.117569 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8fkdw" event={"ID":"0368d478-39bb-4164-8c20-bb8848762359","Type":"ContainerDied","Data":"d55edcc3451a21be5bf2ed3677878e8851e6b869db2604df781a087e5bbaec0e"} Sep 16 18:40:44.793039 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:44.793008 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:44.793226 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:44.793121 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:44.793547 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:44.793530 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:44.793659 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:44.793639 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:45.790938 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:45.790915 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:45.791326 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:45.791029 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:46.123894 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:46.123864 2566 generic.go:356] "Generic (PLEG): container finished" podID="0368d478-39bb-4164-8c20-bb8848762359" containerID="b578c28c383b37e898b1907ced2e857c78fd3f84fdb1e1ad4cb9f425ee7ca545" exitCode=0 Sep 16 18:40:46.124040 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:46.123925 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8fkdw" event={"ID":"0368d478-39bb-4164-8c20-bb8848762359","Type":"ContainerDied","Data":"b578c28c383b37e898b1907ced2e857c78fd3f84fdb1e1ad4cb9f425ee7ca545"} Sep 16 18:40:46.791017 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:46.790989 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:46.791501 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:46.790990 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:46.791501 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:46.791122 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c2scp" podUID="b83a34db-f76e-4f19-b377-32bba0f5fa2b" Sep 16 18:40:46.791501 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:46.791210 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-swmrz" podUID="664c47f5-9768-4395-8396-ce9ac30421e3" Sep 16 18:40:47.791001 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:47.790972 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:47.791238 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:47.791075 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4dw9" podUID="3177af74-fcce-4973-9f87-3dea0506ae3d" Sep 16 18:40:48.423901 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.423870 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:48.424117 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.424004 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:48.424117 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.424060 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret podName:664c47f5-9768-4395-8396-ce9ac30421e3 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:20.42404527 +0000 UTC m=+66.104899893 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret") pod "global-pull-secret-syncer-swmrz" (UID: "664c47f5-9768-4395-8396-ce9ac30421e3") : object "kube-system"/"original-pull-secret" not registered Sep 16 18:40:48.525288 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.525259 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:48.525405 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.525390 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:48.525462 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.525452 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs podName:b83a34db-f76e-4f19-b377-32bba0f5fa2b nodeName:}" failed. No retries permitted until 2026-09-16 18:41:20.525433679 +0000 UTC m=+66.206288295 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs") pod "network-metrics-daemon-c2scp" (UID: "b83a34db-f76e-4f19-b377-32bba0f5fa2b") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 16 18:40:48.614035 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.614007 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-99.ec2.internal" event="NodeReady" Sep 16 18:40:48.614179 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.614150 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 16 18:40:48.625543 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.625515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68\") pod \"network-check-target-s4dw9\" (UID: \"3177af74-fcce-4973-9f87-3dea0506ae3d\") " pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:48.625645 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.625627 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 16 18:40:48.625645 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.625641 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 16 18:40:48.625750 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.625649 2566 projected.go:196] Error preparing data for projected volume kube-api-access-fpp68 for pod openshift-network-diagnostics/network-check-target-s4dw9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:48.625750 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.625696 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68 podName:3177af74-fcce-4973-9f87-3dea0506ae3d nodeName:}" failed. No retries permitted until 2026-09-16 18:41:20.625678731 +0000 UTC m=+66.306533344 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fpp68" (UniqueName: "kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68") pod "network-check-target-s4dw9" (UID: "3177af74-fcce-4973-9f87-3dea0506ae3d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 16 18:40:48.663600 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.663579 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c48d98d7b-hfww6"] Sep 16 18:40:48.668683 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.668663 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.672867 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.672843 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 16 18:40:48.673020 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.672901 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6l2zg\"" Sep 16 18:40:48.673020 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.672920 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 16 18:40:48.673183 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.673166 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 16 18:40:48.679033 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.679010 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 16 18:40:48.697704 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.697682 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fvmvk"] Sep 16 18:40:48.701393 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.701377 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c48d98d7b-hfww6"] Sep 16 18:40:48.701493 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.701475 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:48.704015 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.703996 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 16 18:40:48.704104 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.704051 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 16 18:40:48.704152 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.704108 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9jkmg\"" Sep 16 18:40:48.707925 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.707905 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rm9x6"] Sep 16 18:40:48.711825 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.711809 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rm9x6" Sep 16 18:40:48.713760 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.713745 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 16 18:40:48.714133 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.714114 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 16 18:40:48.714614 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.714298 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 16 18:40:48.715000 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.714981 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xc5gc\"" Sep 16 18:40:48.715341 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.715321 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fvmvk"] Sep 16 18:40:48.721584 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.721565 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rm9x6"] Sep 16 18:40:48.791064 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.791035 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:40:48.791064 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.791052 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:40:48.793663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.793646 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 16 18:40:48.793984 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.793678 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 16 18:40:48.793984 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.793720 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ndh2n\"" Sep 16 18:40:48.827974 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.827951 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-image-registry-private-configuration\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.828085 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.827980 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4vrt\" (UniqueName: \"kubernetes.io/projected/49cff26f-250f-4a66-9156-4ca5778fa510-kube-api-access-c4vrt\") pod \"ingress-canary-rm9x6\" (UID: \"49cff26f-250f-4a66-9156-4ca5778fa510\") " pod="openshift-ingress-canary/ingress-canary-rm9x6" Sep 16 18:40:48.828085 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.828016 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.828085 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.828042 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77n8g\" (UniqueName: \"kubernetes.io/projected/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-kube-api-access-77n8g\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:48.828272 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.828094 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49cff26f-250f-4a66-9156-4ca5778fa510-cert\") pod \"ingress-canary-rm9x6\" (UID: \"49cff26f-250f-4a66-9156-4ca5778fa510\") " pod="openshift-ingress-canary/ingress-canary-rm9x6" Sep 16 18:40:48.828272 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.828124 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-trusted-ca\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.828272 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.828146 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-ca-trust-extracted\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.828272 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.828167 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:48.828272 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.828195 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-certificates\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.828272 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.828215 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-config-volume\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:48.828562 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.828298 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-bound-sa-token\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.828562 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.828319 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-installation-pull-secrets\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.828562 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.828340 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mljrv\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-kube-api-access-mljrv\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.828562 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.828355 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-tmp-dir\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:48.928738 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.928667 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-trusted-ca\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.928738 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.928718 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-ca-trust-extracted\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.928977 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.928743 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:48.928977 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.928774 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-certificates\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.928977 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.928809 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-config-volume\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:48.928977 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.928877 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-bound-sa-token\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.928977 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.928889 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 18:40:48.928977 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.928958 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls podName:2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:49.428937684 +0000 UTC m=+35.109792309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls") pod "dns-default-fvmvk" (UID: "2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46") : secret "dns-default-metrics-tls" not found Sep 16 18:40:48.929317 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.929113 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-installation-pull-secrets\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.929317 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.929148 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mljrv\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-kube-api-access-mljrv\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.929317 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.929172 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-tmp-dir\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:48.929317 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.929215 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-ca-trust-extracted\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.929317 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.929222 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-image-registry-private-configuration\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.929317 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.929272 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4vrt\" (UniqueName: \"kubernetes.io/projected/49cff26f-250f-4a66-9156-4ca5778fa510-kube-api-access-c4vrt\") pod \"ingress-canary-rm9x6\" (UID: \"49cff26f-250f-4a66-9156-4ca5778fa510\") " pod="openshift-ingress-canary/ingress-canary-rm9x6" Sep 16 18:40:48.929317 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.929310 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.929637 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.929329 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77n8g\" (UniqueName: \"kubernetes.io/projected/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-kube-api-access-77n8g\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:48.929637 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.929347 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49cff26f-250f-4a66-9156-4ca5778fa510-cert\") pod \"ingress-canary-rm9x6\" (UID: \"49cff26f-250f-4a66-9156-4ca5778fa510\") " pod="openshift-ingress-canary/ingress-canary-rm9x6" Sep 16 18:40:48.929637 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.929487 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-config-volume\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:48.929775 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.929753 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-tmp-dir\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:48.929862 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.929843 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 16 18:40:48.929923 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.929864 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c48d98d7b-hfww6: secret "image-registry-tls" not found Sep 16 18:40:48.929923 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:48.929910 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls podName:6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:49.429893968 +0000 UTC m=+35.110748601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls") pod "image-registry-c48d98d7b-hfww6" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9") : secret "image-registry-tls" not found Sep 16 18:40:48.930324 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.930090 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-certificates\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.930324 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.930171 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-trusted-ca\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.933707 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.933676 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-image-registry-private-configuration\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.933707 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.933703 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-installation-pull-secrets\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.933860 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.933737 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/49cff26f-250f-4a66-9156-4ca5778fa510-cert\") pod \"ingress-canary-rm9x6\" (UID: \"49cff26f-250f-4a66-9156-4ca5778fa510\") " pod="openshift-ingress-canary/ingress-canary-rm9x6" Sep 16 18:40:48.941145 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.941118 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-bound-sa-token\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:48.942520 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.942496 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4vrt\" (UniqueName: \"kubernetes.io/projected/49cff26f-250f-4a66-9156-4ca5778fa510-kube-api-access-c4vrt\") pod \"ingress-canary-rm9x6\" (UID: \"49cff26f-250f-4a66-9156-4ca5778fa510\") " pod="openshift-ingress-canary/ingress-canary-rm9x6" Sep 16 18:40:48.942720 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.942696 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77n8g\" (UniqueName: \"kubernetes.io/projected/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-kube-api-access-77n8g\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:48.943669 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:48.943647 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mljrv\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-kube-api-access-mljrv\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:49.022104 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:49.022083 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rm9x6" Sep 16 18:40:49.028442 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:49.028414 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vxc8t_28b45ecf-82c1-4811-9f2b-e9cdf59a7816/konnectivity-agent/0.log" limit=1024 Sep 16 18:40:49.270037 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:49.270001 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vxc8t_28b45ecf-82c1-4811-9f2b-e9cdf59a7816/konnectivity-agent/0.log" limit=1024 Sep 16 18:40:49.433699 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:49.433666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:49.433867 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:49.433723 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:49.433867 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:49.433818 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 16 18:40:49.433867 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:49.433837 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c48d98d7b-hfww6: secret "image-registry-tls" not found Sep 16 18:40:49.433867 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:49.433855 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 18:40:49.434026 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:49.433900 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls podName:6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:50.433880864 +0000 UTC m=+36.114735476 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls") pod "image-registry-c48d98d7b-hfww6" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9") : secret "image-registry-tls" not found Sep 16 18:40:49.434026 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:49.433919 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls podName:2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:50.433910752 +0000 UTC m=+36.114765363 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls") pod "dns-default-fvmvk" (UID: "2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46") : secret "dns-default-metrics-tls" not found Sep 16 18:40:49.791762 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:49.791729 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:40:49.796359 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:49.796331 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 16 18:40:49.796700 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:49.796370 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 16 18:40:49.796940 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:49.796910 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5qbcq\"" Sep 16 18:40:50.137256 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:50.137048 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8fkdw" event={"ID":"0368d478-39bb-4164-8c20-bb8848762359","Type":"ContainerStarted","Data":"1547a3cc5e756b7384cc84a35d721b5bcd2acf4662649ad19a1c67f900a8f7cc"} Sep 16 18:40:50.195111 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:50.195082 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rm9x6"] Sep 16 18:40:50.204004 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:40:50.203981 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49cff26f_250f_4a66_9156_4ca5778fa510.slice/crio-b25a242693c455b889d8547a39abcf884488a4efdbeeae07cd7214129e1fff96 WatchSource:0}: Error finding container b25a242693c455b889d8547a39abcf884488a4efdbeeae07cd7214129e1fff96: Status 404 returned error can't find the container with id b25a242693c455b889d8547a39abcf884488a4efdbeeae07cd7214129e1fff96 Sep 16 18:40:50.441496 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:50.441427 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:50.441496 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:50.441473 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:50.441657 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:50.441568 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 16 18:40:50.441657 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:50.441586 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c48d98d7b-hfww6: secret "image-registry-tls" not found Sep 16 18:40:50.441657 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:50.441599 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 18:40:50.441657 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:50.441636 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls podName:6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:52.441620364 +0000 UTC m=+38.122474977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls") pod "image-registry-c48d98d7b-hfww6" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9") : secret "image-registry-tls" not found Sep 16 18:40:50.441657 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:50.441651 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls podName:2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:52.441644547 +0000 UTC m=+38.122499156 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls") pod "dns-default-fvmvk" (UID: "2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46") : secret "dns-default-metrics-tls" not found Sep 16 18:40:51.146418 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:51.146379 2566 generic.go:356] "Generic (PLEG): container finished" podID="0368d478-39bb-4164-8c20-bb8848762359" containerID="1547a3cc5e756b7384cc84a35d721b5bcd2acf4662649ad19a1c67f900a8f7cc" exitCode=0 Sep 16 18:40:51.147206 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:51.146457 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8fkdw" event={"ID":"0368d478-39bb-4164-8c20-bb8848762359","Type":"ContainerDied","Data":"1547a3cc5e756b7384cc84a35d721b5bcd2acf4662649ad19a1c67f900a8f7cc"} Sep 16 18:40:51.147816 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:51.147788 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rm9x6" event={"ID":"49cff26f-250f-4a66-9156-4ca5778fa510","Type":"ContainerStarted","Data":"b25a242693c455b889d8547a39abcf884488a4efdbeeae07cd7214129e1fff96"} Sep 16 18:40:52.154296 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:52.154260 2566 generic.go:356] "Generic (PLEG): container finished" podID="0368d478-39bb-4164-8c20-bb8848762359" containerID="7ea26a1ba6086799eec914445be84fb349a51051f59805ea4abf8f6c32768d6c" exitCode=0 Sep 16 18:40:52.154881 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:52.154304 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8fkdw" event={"ID":"0368d478-39bb-4164-8c20-bb8848762359","Type":"ContainerDied","Data":"7ea26a1ba6086799eec914445be84fb349a51051f59805ea4abf8f6c32768d6c"} Sep 16 18:40:52.458378 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:52.458110 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:52.458472 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:52.458401 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:52.458472 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:52.458299 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 16 18:40:52.458472 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:52.458455 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c48d98d7b-hfww6: secret "image-registry-tls" not found Sep 16 18:40:52.458614 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:52.458512 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls podName:6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:56.458489911 +0000 UTC m=+42.139344537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls") pod "image-registry-c48d98d7b-hfww6" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9") : secret "image-registry-tls" not found Sep 16 18:40:52.458614 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:52.458558 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 18:40:52.458719 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:52.458617 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls podName:2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46 nodeName:}" failed. No retries permitted until 2026-09-16 18:40:56.458605448 +0000 UTC m=+42.139460072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls") pod "dns-default-fvmvk" (UID: "2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46") : secret "dns-default-metrics-tls" not found Sep 16 18:40:53.160094 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:53.160020 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8fkdw" event={"ID":"0368d478-39bb-4164-8c20-bb8848762359","Type":"ContainerStarted","Data":"69ab6e8aa1d31506eb63a7813597e27b615e47914fe1d9985573c7f20e02f2b5"} Sep 16 18:40:53.161440 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:53.161416 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rm9x6" event={"ID":"49cff26f-250f-4a66-9156-4ca5778fa510","Type":"ContainerStarted","Data":"c3888729eb6955cb1d5f36b2242ba02bd6fa1daca58504a859b44206eb5680c4"} Sep 16 18:40:53.211228 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:53.211161 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8fkdw" podStartSLOduration=9.776086945 podStartE2EDuration="38.2111471s" totalImagesPullingTime="28.435060155s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:53.210179231 +0000 UTC m=+38.891033896" watchObservedRunningTime="2026-09-16 18:40:53.2111471 +0000 UTC m=+38.892001731" Sep 16 18:40:53.239789 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:53.239749 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rm9x6" podStartSLOduration=2.979901212 podStartE2EDuration="5.239734242s" totalImagesPullingTime="2.25983303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:40:53.239425603 +0000 UTC m=+38.920280236" watchObservedRunningTime="2026-09-16 18:40:53.239734242 +0000 UTC m=+38.920588876" Sep 16 18:40:56.485421 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:56.485380 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:40:56.485421 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:56.485427 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:40:56.485919 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:56.485518 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 18:40:56.485919 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:56.485542 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 16 18:40:56.485919 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:56.485566 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c48d98d7b-hfww6: secret "image-registry-tls" not found Sep 16 18:40:56.485919 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:56.485567 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls podName:2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:04.485554234 +0000 UTC m=+50.166408843 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls") pod "dns-default-fvmvk" (UID: "2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46") : secret "dns-default-metrics-tls" not found Sep 16 18:40:56.485919 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:56.485631 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls podName:6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:04.485611118 +0000 UTC m=+50.166465740 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls") pod "image-registry-c48d98d7b-hfww6" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9") : secret "image-registry-tls" not found Sep 16 18:40:59.475315 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.475279 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-qn2k4"] Sep 16 18:40:59.501145 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.501110 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx"] Sep 16 18:40:59.501345 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.501279 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-qn2k4" Sep 16 18:40:59.503716 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.503690 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-mx27d\"" Sep 16 18:40:59.517065 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.517043 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb"] Sep 16 18:40:59.517166 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.517148 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" Sep 16 18:40:59.519612 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.519583 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 16 18:40:59.519774 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.519621 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 16 18:40:59.519774 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.519583 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 16 18:40:59.519774 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.519670 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 16 18:40:59.519774 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.519730 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kzzwz\"" Sep 16 18:40:59.541081 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.541064 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-n92m9"] Sep 16 18:40:59.541196 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.541169 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" Sep 16 18:40:59.543468 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.543433 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 16 18:40:59.543586 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.543495 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 16 18:40:59.543900 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.543884 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-zvww9\"" Sep 16 18:40:59.544277 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.544263 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 16 18:40:59.544327 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.544286 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 16 18:40:59.565169 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.565151 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-qn2k4"] Sep 16 18:40:59.565288 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.565177 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb"] Sep 16 18:40:59.565288 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.565210 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx"] Sep 16 18:40:59.565288 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.565231 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-n92m9"] Sep 16 18:40:59.565385 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.565298 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.567398 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.567376 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 16 18:40:59.567508 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.567461 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 16 18:40:59.568009 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.567993 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 16 18:40:59.568115 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.568101 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 16 18:40:59.568181 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.568162 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bgtkm\"" Sep 16 18:40:59.574955 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.574937 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 16 18:40:59.605218 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.605199 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27de7a26-8b7b-4003-8b4e-83232c28c32d-serving-cert\") pod \"service-ca-operator-cf7c776b7-gzxqx\" (UID: \"27de7a26-8b7b-4003-8b4e-83232c28c32d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" Sep 16 18:40:59.605279 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.605253 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg566\" (UniqueName: \"kubernetes.io/projected/8a8bc40c-0ed0-45ad-88a9-91c0e51a12fb-kube-api-access-xg566\") pod \"network-check-source-6f8b768455-qn2k4\" (UID: \"8a8bc40c-0ed0-45ad-88a9-91c0e51a12fb\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-qn2k4" Sep 16 18:40:59.605316 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.605293 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27de7a26-8b7b-4003-8b4e-83232c28c32d-config\") pod \"service-ca-operator-cf7c776b7-gzxqx\" (UID: \"27de7a26-8b7b-4003-8b4e-83232c28c32d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" Sep 16 18:40:59.605316 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.605311 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-558fg\" (UniqueName: \"kubernetes.io/projected/27de7a26-8b7b-4003-8b4e-83232c28c32d-kube-api-access-558fg\") pod \"service-ca-operator-cf7c776b7-gzxqx\" (UID: \"27de7a26-8b7b-4003-8b4e-83232c28c32d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" Sep 16 18:40:59.683492 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.683465 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj"] Sep 16 18:40:59.701213 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.701178 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jtvst"] Sep 16 18:40:59.701300 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.701261 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:40:59.703558 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.703487 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 16 18:40:59.703663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.703639 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 16 18:40:59.704164 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.704141 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 16 18:40:59.704376 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.704285 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-l7txb\"" Sep 16 18:40:59.704376 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.704145 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 16 18:40:59.705562 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.705543 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-558fg\" (UniqueName: \"kubernetes.io/projected/27de7a26-8b7b-4003-8b4e-83232c28c32d-kube-api-access-558fg\") pod \"service-ca-operator-cf7c776b7-gzxqx\" (UID: \"27de7a26-8b7b-4003-8b4e-83232c28c32d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" Sep 16 18:40:59.705655 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.705577 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2p9g\" (UniqueName: \"kubernetes.io/projected/836dca5e-9f3b-4d19-9086-6453deee99e4-kube-api-access-v2p9g\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nvnqb\" (UID: \"836dca5e-9f3b-4d19-9086-6453deee99e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" Sep 16 18:40:59.705655 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.705633 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27de7a26-8b7b-4003-8b4e-83232c28c32d-serving-cert\") pod \"service-ca-operator-cf7c776b7-gzxqx\" (UID: \"27de7a26-8b7b-4003-8b4e-83232c28c32d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" Sep 16 18:40:59.705762 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.705681 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be5b5e3e-2d54-4755-bf1e-307ebaa84ad3-serving-cert\") pod \"console-operator-696884fb9c-n92m9\" (UID: \"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3\") " pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.705829 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.705809 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be5b5e3e-2d54-4755-bf1e-307ebaa84ad3-trusted-ca\") pod \"console-operator-696884fb9c-n92m9\" (UID: \"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3\") " pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.705883 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.705870 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be5b5e3e-2d54-4755-bf1e-307ebaa84ad3-config\") pod \"console-operator-696884fb9c-n92m9\" (UID: \"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3\") " pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.705944 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.705929 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc5cn\" (UniqueName: \"kubernetes.io/projected/be5b5e3e-2d54-4755-bf1e-307ebaa84ad3-kube-api-access-jc5cn\") pod \"console-operator-696884fb9c-n92m9\" (UID: \"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3\") " pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.705996 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.705973 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg566\" (UniqueName: \"kubernetes.io/projected/8a8bc40c-0ed0-45ad-88a9-91c0e51a12fb-kube-api-access-xg566\") pod \"network-check-source-6f8b768455-qn2k4\" (UID: \"8a8bc40c-0ed0-45ad-88a9-91c0e51a12fb\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-qn2k4" Sep 16 18:40:59.706051 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.706028 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27de7a26-8b7b-4003-8b4e-83232c28c32d-config\") pod \"service-ca-operator-cf7c776b7-gzxqx\" (UID: \"27de7a26-8b7b-4003-8b4e-83232c28c32d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" Sep 16 18:40:59.706101 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.706055 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/836dca5e-9f3b-4d19-9086-6453deee99e4-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nvnqb\" (UID: \"836dca5e-9f3b-4d19-9086-6453deee99e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" Sep 16 18:40:59.706101 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.706095 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/836dca5e-9f3b-4d19-9086-6453deee99e4-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nvnqb\" (UID: \"836dca5e-9f3b-4d19-9086-6453deee99e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" Sep 16 18:40:59.706622 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.706599 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/27de7a26-8b7b-4003-8b4e-83232c28c32d-config\") pod \"service-ca-operator-cf7c776b7-gzxqx\" (UID: \"27de7a26-8b7b-4003-8b4e-83232c28c32d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" Sep 16 18:40:59.709544 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.709524 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/27de7a26-8b7b-4003-8b4e-83232c28c32d-serving-cert\") pod \"service-ca-operator-cf7c776b7-gzxqx\" (UID: \"27de7a26-8b7b-4003-8b4e-83232c28c32d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" Sep 16 18:40:59.719343 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.719323 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-558fg\" (UniqueName: \"kubernetes.io/projected/27de7a26-8b7b-4003-8b4e-83232c28c32d-kube-api-access-558fg\") pod \"service-ca-operator-cf7c776b7-gzxqx\" (UID: \"27de7a26-8b7b-4003-8b4e-83232c28c32d\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" Sep 16 18:40:59.720144 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.720127 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jtvst"] Sep 16 18:40:59.720211 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.720154 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj"] Sep 16 18:40:59.720211 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.720169 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-dz57d"] Sep 16 18:40:59.720315 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.720255 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jtvst" Sep 16 18:40:59.722554 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.722465 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 16 18:40:59.722654 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.722579 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zqlpk\"" Sep 16 18:40:59.722755 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.722702 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 16 18:40:59.722835 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.722772 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg566\" (UniqueName: \"kubernetes.io/projected/8a8bc40c-0ed0-45ad-88a9-91c0e51a12fb-kube-api-access-xg566\") pod \"network-check-source-6f8b768455-qn2k4\" (UID: \"8a8bc40c-0ed0-45ad-88a9-91c0e51a12fb\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-qn2k4" Sep 16 18:40:59.740866 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.740847 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-54bb988d4b-wn9jr"] Sep 16 18:40:59.740985 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.740970 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:40:59.742978 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.742952 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 16 18:40:59.743065 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.743005 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 16 18:40:59.743206 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.743169 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-jvcng\"" Sep 16 18:40:59.743288 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.743239 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 16 18:40:59.743437 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.743418 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 16 18:40:59.748218 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.748201 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 16 18:40:59.760122 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.760103 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8"] Sep 16 18:40:59.760251 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.760234 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:40:59.762338 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.762320 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g8fnt\"" Sep 16 18:40:59.762516 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.762495 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 16 18:40:59.762709 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.762690 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 16 18:40:59.762709 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.762708 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 16 18:40:59.762822 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.762710 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 16 18:40:59.763264 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.763248 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 16 18:40:59.763334 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.763305 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 16 18:40:59.782514 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.782496 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-dz57d"] Sep 16 18:40:59.782600 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.782524 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8"] Sep 16 18:40:59.782600 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.782541 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54bb988d4b-wn9jr"] Sep 16 18:40:59.782674 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.782607 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:40:59.784678 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.784659 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 16 18:40:59.784758 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.784713 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 16 18:40:59.784813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.784759 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-p4q2n\"" Sep 16 18:40:59.784813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.784801 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 16 18:40:59.806424 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.806406 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be5b5e3e-2d54-4755-bf1e-307ebaa84ad3-config\") pod \"console-operator-696884fb9c-n92m9\" (UID: \"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3\") " pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.806499 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.806438 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/836dca5e-9f3b-4d19-9086-6453deee99e4-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nvnqb\" (UID: \"836dca5e-9f3b-4d19-9086-6453deee99e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" Sep 16 18:40:59.806499 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.806457 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2p9g\" (UniqueName: \"kubernetes.io/projected/836dca5e-9f3b-4d19-9086-6453deee99e4-kube-api-access-v2p9g\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nvnqb\" (UID: \"836dca5e-9f3b-4d19-9086-6453deee99e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" Sep 16 18:40:59.806499 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.806494 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/836dca5e-9f3b-4d19-9086-6453deee99e4-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nvnqb\" (UID: \"836dca5e-9f3b-4d19-9086-6453deee99e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" Sep 16 18:40:59.806694 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.806672 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlh44\" (UniqueName: \"kubernetes.io/projected/9c7b5a24-e9ce-4270-830a-b1a4882df402-kube-api-access-hlh44\") pod \"volume-data-source-validator-86695b7cc7-jtvst\" (UID: \"9c7b5a24-e9ce-4270-830a-b1a4882df402\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jtvst" Sep 16 18:40:59.806765 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.806747 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be5b5e3e-2d54-4755-bf1e-307ebaa84ad3-serving-cert\") pod \"console-operator-696884fb9c-n92m9\" (UID: \"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3\") " pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.806802 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.806786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be5b5e3e-2d54-4755-bf1e-307ebaa84ad3-trusted-ca\") pod \"console-operator-696884fb9c-n92m9\" (UID: \"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3\") " pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.806851 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.806816 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/02248648-cbdf-4352-b6f7-c91ecd16ab6a-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:40:59.806851 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.806843 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7qtc\" (UniqueName: \"kubernetes.io/projected/02248648-cbdf-4352-b6f7-c91ecd16ab6a-kube-api-access-r7qtc\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:40:59.806948 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.806890 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc5cn\" (UniqueName: \"kubernetes.io/projected/be5b5e3e-2d54-4755-bf1e-307ebaa84ad3-kube-api-access-jc5cn\") pod \"console-operator-696884fb9c-n92m9\" (UID: \"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3\") " pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.806998 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.806946 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:40:59.807050 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.807020 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/836dca5e-9f3b-4d19-9086-6453deee99e4-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nvnqb\" (UID: \"836dca5e-9f3b-4d19-9086-6453deee99e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" Sep 16 18:40:59.807278 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.807255 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/be5b5e3e-2d54-4755-bf1e-307ebaa84ad3-config\") pod \"console-operator-696884fb9c-n92m9\" (UID: \"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3\") " pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.807789 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.807769 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/be5b5e3e-2d54-4755-bf1e-307ebaa84ad3-trusted-ca\") pod \"console-operator-696884fb9c-n92m9\" (UID: \"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3\") " pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.808947 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.808923 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/be5b5e3e-2d54-4755-bf1e-307ebaa84ad3-serving-cert\") pod \"console-operator-696884fb9c-n92m9\" (UID: \"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3\") " pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.808947 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.808933 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/836dca5e-9f3b-4d19-9086-6453deee99e4-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nvnqb\" (UID: \"836dca5e-9f3b-4d19-9086-6453deee99e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" Sep 16 18:40:59.810952 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.810937 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-qn2k4" Sep 16 18:40:59.817124 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.817095 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2p9g\" (UniqueName: \"kubernetes.io/projected/836dca5e-9f3b-4d19-9086-6453deee99e4-kube-api-access-v2p9g\") pod \"kube-storage-version-migrator-operator-6c8f84f897-nvnqb\" (UID: \"836dca5e-9f3b-4d19-9086-6453deee99e4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" Sep 16 18:40:59.817335 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.817320 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc5cn\" (UniqueName: \"kubernetes.io/projected/be5b5e3e-2d54-4755-bf1e-307ebaa84ad3-kube-api-access-jc5cn\") pod \"console-operator-696884fb9c-n92m9\" (UID: \"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3\") " pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.825325 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.825310 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" Sep 16 18:40:59.849494 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.849477 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" Sep 16 18:40:59.873904 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.873883 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:40:59.907499 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907460 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nfl4\" (UniqueName: \"kubernetes.io/projected/3ff4c1db-9b70-4dae-977e-85493cace17a-kube-api-access-7nfl4\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:40:59.907641 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907533 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:40:59.907641 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907575 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ff4c1db-9b70-4dae-977e-85493cace17a-tmp\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:40:59.907641 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907602 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5d86\" (UniqueName: \"kubernetes.io/projected/e4e71011-5834-4c22-a24b-8d0ff2e90359-kube-api-access-z5d86\") pod \"cluster-samples-operator-86499c89c4-zngz8\" (UID: \"e4e71011-5834-4c22-a24b-8d0ff2e90359\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907668 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlh44\" (UniqueName: \"kubernetes.io/projected/9c7b5a24-e9ce-4270-830a-b1a4882df402-kube-api-access-hlh44\") pod \"volume-data-source-validator-86695b7cc7-jtvst\" (UID: \"9c7b5a24-e9ce-4270-830a-b1a4882df402\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jtvst" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907699 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ff4c1db-9b70-4dae-977e-85493cace17a-serving-cert\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-default-certificate\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907760 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-zngz8\" (UID: \"e4e71011-5834-4c22-a24b-8d0ff2e90359\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907798 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/02248648-cbdf-4352-b6f7-c91ecd16ab6a-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907824 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7qtc\" (UniqueName: \"kubernetes.io/projected/02248648-cbdf-4352-b6f7-c91ecd16ab6a-kube-api-access-r7qtc\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907869 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqlkb\" (UniqueName: \"kubernetes.io/projected/40180743-5519-4b63-8b3b-e790ef62c2be-kube-api-access-lqlkb\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907895 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff4c1db-9b70-4dae-977e-85493cace17a-trusted-ca-bundle\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907917 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-stats-auth\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907943 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.907974 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3ff4c1db-9b70-4dae-977e-85493cace17a-snapshots\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.908019 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff4c1db-9b70-4dae-977e-85493cace17a-service-ca-bundle\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.908045 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:59.908213 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 16 18:40:59.908663 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:40:59.908269 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls podName:02248648-cbdf-4352-b6f7-c91ecd16ab6a nodeName:}" failed. No retries permitted until 2026-09-16 18:41:00.408250399 +0000 UTC m=+46.089105014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-sw7rj" (UID: "02248648-cbdf-4352-b6f7-c91ecd16ab6a") : secret "cluster-monitoring-operator-tls" not found Sep 16 18:40:59.909802 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.908820 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/02248648-cbdf-4352-b6f7-c91ecd16ab6a-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:40:59.928922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.928864 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlh44\" (UniqueName: \"kubernetes.io/projected/9c7b5a24-e9ce-4270-830a-b1a4882df402-kube-api-access-hlh44\") pod \"volume-data-source-validator-86695b7cc7-jtvst\" (UID: \"9c7b5a24-e9ce-4270-830a-b1a4882df402\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jtvst" Sep 16 18:40:59.929463 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:40:59.929441 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7qtc\" (UniqueName: \"kubernetes.io/projected/02248648-cbdf-4352-b6f7-c91ecd16ab6a-kube-api-access-r7qtc\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:41:00.008396 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008371 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ff4c1db-9b70-4dae-977e-85493cace17a-tmp\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.008503 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008405 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5d86\" (UniqueName: \"kubernetes.io/projected/e4e71011-5834-4c22-a24b-8d0ff2e90359-kube-api-access-z5d86\") pod \"cluster-samples-operator-86499c89c4-zngz8\" (UID: \"e4e71011-5834-4c22-a24b-8d0ff2e90359\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:41:00.008545 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008536 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ff4c1db-9b70-4dae-977e-85493cace17a-serving-cert\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.008585 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008554 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-default-certificate\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:00.008585 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008578 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-zngz8\" (UID: \"e4e71011-5834-4c22-a24b-8d0ff2e90359\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:41:00.008677 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008613 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqlkb\" (UniqueName: \"kubernetes.io/projected/40180743-5519-4b63-8b3b-e790ef62c2be-kube-api-access-lqlkb\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:00.008677 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008639 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff4c1db-9b70-4dae-977e-85493cace17a-trusted-ca-bundle\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.008677 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-stats-auth\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:00.008816 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008702 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3ff4c1db-9b70-4dae-977e-85493cace17a-snapshots\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.008816 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008740 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff4c1db-9b70-4dae-977e-85493cace17a-service-ca-bundle\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.008816 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:00.008816 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008788 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nfl4\" (UniqueName: \"kubernetes.io/projected/3ff4c1db-9b70-4dae-977e-85493cace17a-kube-api-access-7nfl4\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.009005 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:00.009005 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.008701 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3ff4c1db-9b70-4dae-977e-85493cace17a-tmp\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.009005 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:00.008932 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 16 18:41:00.009005 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:00.008985 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs podName:40180743-5519-4b63-8b3b-e790ef62c2be nodeName:}" failed. No retries permitted until 2026-09-16 18:41:00.508965653 +0000 UTC m=+46.189820277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs") pod "router-default-54bb988d4b-wn9jr" (UID: "40180743-5519-4b63-8b3b-e790ef62c2be") : secret "router-metrics-certs-default" not found Sep 16 18:41:00.009220 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:00.009123 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle podName:40180743-5519-4b63-8b3b-e790ef62c2be nodeName:}" failed. No retries permitted until 2026-09-16 18:41:00.509091009 +0000 UTC m=+46.189945637 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle") pod "router-default-54bb988d4b-wn9jr" (UID: "40180743-5519-4b63-8b3b-e790ef62c2be") : configmap references non-existent config key: service-ca.crt Sep 16 18:41:00.009287 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:00.009249 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 16 18:41:00.009344 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:00.009293 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls podName:e4e71011-5834-4c22-a24b-8d0ff2e90359 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:00.509279838 +0000 UTC m=+46.190134463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-zngz8" (UID: "e4e71011-5834-4c22-a24b-8d0ff2e90359") : secret "samples-operator-tls" not found Sep 16 18:41:00.009676 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.009502 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/3ff4c1db-9b70-4dae-977e-85493cace17a-snapshots\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.009676 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.009628 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff4c1db-9b70-4dae-977e-85493cace17a-service-ca-bundle\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.009802 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.009682 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ff4c1db-9b70-4dae-977e-85493cace17a-trusted-ca-bundle\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.010872 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.010851 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-default-certificate\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:00.011376 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.011356 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3ff4c1db-9b70-4dae-977e-85493cace17a-serving-cert\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.011439 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.011394 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-stats-auth\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:00.018058 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.018025 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5d86\" (UniqueName: \"kubernetes.io/projected/e4e71011-5834-4c22-a24b-8d0ff2e90359-kube-api-access-z5d86\") pod \"cluster-samples-operator-86499c89c4-zngz8\" (UID: \"e4e71011-5834-4c22-a24b-8d0ff2e90359\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:41:00.018615 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.018597 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqlkb\" (UniqueName: \"kubernetes.io/projected/40180743-5519-4b63-8b3b-e790ef62c2be-kube-api-access-lqlkb\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:00.018933 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.018914 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nfl4\" (UniqueName: \"kubernetes.io/projected/3ff4c1db-9b70-4dae-977e-85493cace17a-kube-api-access-7nfl4\") pod \"insights-operator-5844b856db-dz57d\" (UID: \"3ff4c1db-9b70-4dae-977e-85493cace17a\") " pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.035653 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.035634 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jtvst" Sep 16 18:41:00.049830 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.049812 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-dz57d" Sep 16 18:41:00.122031 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.122002 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-qn2k4"] Sep 16 18:41:00.127033 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:00.127005 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a8bc40c_0ed0_45ad_88a9_91c0e51a12fb.slice/crio-83aa7d82b6dad0e02af7c500a546bcd795d9ba3c3f16a53411adf223efedbd60 WatchSource:0}: Error finding container 83aa7d82b6dad0e02af7c500a546bcd795d9ba3c3f16a53411adf223efedbd60: Status 404 returned error can't find the container with id 83aa7d82b6dad0e02af7c500a546bcd795d9ba3c3f16a53411adf223efedbd60 Sep 16 18:41:00.146948 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.146920 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx"] Sep 16 18:41:00.150069 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:00.150043 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27de7a26_8b7b_4003_8b4e_83232c28c32d.slice/crio-636d598cecd313e49f55d16980e421c9474e230110d161bf64f492a0425bde94 WatchSource:0}: Error finding container 636d598cecd313e49f55d16980e421c9474e230110d161bf64f492a0425bde94: Status 404 returned error can't find the container with id 636d598cecd313e49f55d16980e421c9474e230110d161bf64f492a0425bde94 Sep 16 18:41:00.172063 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.172029 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb"] Sep 16 18:41:00.178625 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:00.178588 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod836dca5e_9f3b_4d19_9086_6453deee99e4.slice/crio-849da264bc79baf0b4b602204445a3c121805969b1d6ea4f0021634abde7fcae WatchSource:0}: Error finding container 849da264bc79baf0b4b602204445a3c121805969b1d6ea4f0021634abde7fcae: Status 404 returned error can't find the container with id 849da264bc79baf0b4b602204445a3c121805969b1d6ea4f0021634abde7fcae Sep 16 18:41:00.180797 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.180725 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" event={"ID":"27de7a26-8b7b-4003-8b4e-83232c28c32d","Type":"ContainerStarted","Data":"636d598cecd313e49f55d16980e421c9474e230110d161bf64f492a0425bde94"} Sep 16 18:41:00.182061 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.182026 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-qn2k4" event={"ID":"8a8bc40c-0ed0-45ad-88a9-91c0e51a12fb","Type":"ContainerStarted","Data":"83aa7d82b6dad0e02af7c500a546bcd795d9ba3c3f16a53411adf223efedbd60"} Sep 16 18:41:00.205528 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.205509 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-n92m9"] Sep 16 18:41:00.208095 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:00.208076 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe5b5e3e_2d54_4755_bf1e_307ebaa84ad3.slice/crio-9454cd680aa030b72f14694ce7df9586a2c13f96e7e96de35241742619c48a22 WatchSource:0}: Error finding container 9454cd680aa030b72f14694ce7df9586a2c13f96e7e96de35241742619c48a22: Status 404 returned error can't find the container with id 9454cd680aa030b72f14694ce7df9586a2c13f96e7e96de35241742619c48a22 Sep 16 18:41:00.346373 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.346324 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jtvst"] Sep 16 18:41:00.348947 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:00.348926 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c7b5a24_e9ce_4270_830a_b1a4882df402.slice/crio-5e7d2c6e1611db74e26186342cd27c3bab078d7f338645749d56edf4b773a9a8 WatchSource:0}: Error finding container 5e7d2c6e1611db74e26186342cd27c3bab078d7f338645749d56edf4b773a9a8: Status 404 returned error can't find the container with id 5e7d2c6e1611db74e26186342cd27c3bab078d7f338645749d56edf4b773a9a8 Sep 16 18:41:00.400132 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.400110 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-dz57d"] Sep 16 18:41:00.401407 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:00.401388 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff4c1db_9b70_4dae_977e_85493cace17a.slice/crio-2f9b7c198bb0fff0320fe54a4216591e6f7df85c760c8db3bdcdb7eef1591ae3 WatchSource:0}: Error finding container 2f9b7c198bb0fff0320fe54a4216591e6f7df85c760c8db3bdcdb7eef1591ae3: Status 404 returned error can't find the container with id 2f9b7c198bb0fff0320fe54a4216591e6f7df85c760c8db3bdcdb7eef1591ae3 Sep 16 18:41:00.410913 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.410894 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:41:00.411007 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:00.410997 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 16 18:41:00.411051 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:00.411037 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls podName:02248648-cbdf-4352-b6f7-c91ecd16ab6a nodeName:}" failed. No retries permitted until 2026-09-16 18:41:01.411024493 +0000 UTC m=+47.091879103 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-sw7rj" (UID: "02248648-cbdf-4352-b6f7-c91ecd16ab6a") : secret "cluster-monitoring-operator-tls" not found Sep 16 18:41:00.512070 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.512045 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:00.512362 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.512084 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:00.512362 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:00.512133 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-zngz8\" (UID: \"e4e71011-5834-4c22-a24b-8d0ff2e90359\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:41:00.512362 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:00.512218 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle podName:40180743-5519-4b63-8b3b-e790ef62c2be nodeName:}" failed. No retries permitted until 2026-09-16 18:41:01.512182571 +0000 UTC m=+47.193037182 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle") pod "router-default-54bb988d4b-wn9jr" (UID: "40180743-5519-4b63-8b3b-e790ef62c2be") : configmap references non-existent config key: service-ca.crt Sep 16 18:41:00.512362 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:00.512255 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 16 18:41:00.512362 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:00.512272 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 16 18:41:00.512362 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:00.512296 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls podName:e4e71011-5834-4c22-a24b-8d0ff2e90359 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:01.512286375 +0000 UTC m=+47.193140999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-zngz8" (UID: "e4e71011-5834-4c22-a24b-8d0ff2e90359") : secret "samples-operator-tls" not found Sep 16 18:41:00.512362 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:00.512314 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs podName:40180743-5519-4b63-8b3b-e790ef62c2be nodeName:}" failed. No retries permitted until 2026-09-16 18:41:01.512303192 +0000 UTC m=+47.193157821 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs") pod "router-default-54bb988d4b-wn9jr" (UID: "40180743-5519-4b63-8b3b-e790ef62c2be") : secret "router-metrics-certs-default" not found Sep 16 18:41:01.189437 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:01.189309 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-dz57d" event={"ID":"3ff4c1db-9b70-4dae-977e-85493cace17a","Type":"ContainerStarted","Data":"2f9b7c198bb0fff0320fe54a4216591e6f7df85c760c8db3bdcdb7eef1591ae3"} Sep 16 18:41:01.192165 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:01.192074 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" event={"ID":"836dca5e-9f3b-4d19-9086-6453deee99e4","Type":"ContainerStarted","Data":"849da264bc79baf0b4b602204445a3c121805969b1d6ea4f0021634abde7fcae"} Sep 16 18:41:01.194428 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:01.194335 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jtvst" event={"ID":"9c7b5a24-e9ce-4270-830a-b1a4882df402","Type":"ContainerStarted","Data":"5e7d2c6e1611db74e26186342cd27c3bab078d7f338645749d56edf4b773a9a8"} Sep 16 18:41:01.195954 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:01.195906 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" event={"ID":"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3","Type":"ContainerStarted","Data":"9454cd680aa030b72f14694ce7df9586a2c13f96e7e96de35241742619c48a22"} Sep 16 18:41:01.421778 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:01.421222 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:41:01.421778 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:01.421373 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 16 18:41:01.421778 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:01.421437 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls podName:02248648-cbdf-4352-b6f7-c91ecd16ab6a nodeName:}" failed. No retries permitted until 2026-09-16 18:41:03.421416849 +0000 UTC m=+49.102271483 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-sw7rj" (UID: "02248648-cbdf-4352-b6f7-c91ecd16ab6a") : secret "cluster-monitoring-operator-tls" not found Sep 16 18:41:01.521914 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:01.521859 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:01.522386 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:01.521928 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:01.522386 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:01.522025 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-zngz8\" (UID: \"e4e71011-5834-4c22-a24b-8d0ff2e90359\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:41:01.522386 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:01.522063 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle podName:40180743-5519-4b63-8b3b-e790ef62c2be nodeName:}" failed. No retries permitted until 2026-09-16 18:41:03.522031545 +0000 UTC m=+49.202886176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle") pod "router-default-54bb988d4b-wn9jr" (UID: "40180743-5519-4b63-8b3b-e790ef62c2be") : configmap references non-existent config key: service-ca.crt Sep 16 18:41:01.522386 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:01.522121 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 16 18:41:01.522386 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:01.522175 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls podName:e4e71011-5834-4c22-a24b-8d0ff2e90359 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:03.522157482 +0000 UTC m=+49.203012105 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-zngz8" (UID: "e4e71011-5834-4c22-a24b-8d0ff2e90359") : secret "samples-operator-tls" not found Sep 16 18:41:01.522386 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:01.522265 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 16 18:41:01.522386 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:01.522304 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs podName:40180743-5519-4b63-8b3b-e790ef62c2be nodeName:}" failed. No retries permitted until 2026-09-16 18:41:03.522293093 +0000 UTC m=+49.203147728 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs") pod "router-default-54bb988d4b-wn9jr" (UID: "40180743-5519-4b63-8b3b-e790ef62c2be") : secret "router-metrics-certs-default" not found Sep 16 18:41:03.442833 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:03.442793 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:41:03.443251 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:03.442942 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 16 18:41:03.443251 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:03.443008 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls podName:02248648-cbdf-4352-b6f7-c91ecd16ab6a nodeName:}" failed. No retries permitted until 2026-09-16 18:41:07.442992796 +0000 UTC m=+53.123847411 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-sw7rj" (UID: "02248648-cbdf-4352-b6f7-c91ecd16ab6a") : secret "cluster-monitoring-operator-tls" not found Sep 16 18:41:03.543728 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:03.543694 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:03.543893 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:03.543738 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:03.543893 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:03.543846 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 16 18:41:03.544056 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:03.543890 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle podName:40180743-5519-4b63-8b3b-e790ef62c2be nodeName:}" failed. No retries permitted until 2026-09-16 18:41:07.543868513 +0000 UTC m=+53.224723134 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle") pod "router-default-54bb988d4b-wn9jr" (UID: "40180743-5519-4b63-8b3b-e790ef62c2be") : configmap references non-existent config key: service-ca.crt Sep 16 18:41:03.544056 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:03.543919 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs podName:40180743-5519-4b63-8b3b-e790ef62c2be nodeName:}" failed. No retries permitted until 2026-09-16 18:41:07.543908258 +0000 UTC m=+53.224762867 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs") pod "router-default-54bb988d4b-wn9jr" (UID: "40180743-5519-4b63-8b3b-e790ef62c2be") : secret "router-metrics-certs-default" not found Sep 16 18:41:03.544056 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:03.543959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-zngz8\" (UID: \"e4e71011-5834-4c22-a24b-8d0ff2e90359\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:41:03.544209 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:03.544080 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 16 18:41:03.544209 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:03.544124 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls podName:e4e71011-5834-4c22-a24b-8d0ff2e90359 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:07.54411137 +0000 UTC m=+53.224965983 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-zngz8" (UID: "e4e71011-5834-4c22-a24b-8d0ff2e90359") : secret "samples-operator-tls" not found Sep 16 18:41:04.551692 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:04.551654 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:41:04.552109 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:04.551704 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:41:04.552109 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:04.551818 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 16 18:41:04.552109 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:04.551828 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 16 18:41:04.552109 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:04.551854 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c48d98d7b-hfww6: secret "image-registry-tls" not found Sep 16 18:41:04.552109 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:04.551880 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls podName:2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:20.551867394 +0000 UTC m=+66.232722003 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls") pod "dns-default-fvmvk" (UID: "2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46") : secret "dns-default-metrics-tls" not found Sep 16 18:41:04.552109 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:04.551905 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls podName:6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:20.55188922 +0000 UTC m=+66.232743840 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls") pod "image-registry-c48d98d7b-hfww6" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9") : secret "image-registry-tls" not found Sep 16 18:41:07.216020 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.215985 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-qn2k4" event={"ID":"8a8bc40c-0ed0-45ad-88a9-91c0e51a12fb","Type":"ContainerStarted","Data":"9d2b1baeeb38b0961783643004951e810db66b1205a6a40d3656688ce92d5206"} Sep 16 18:41:07.221146 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.221121 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jtvst" event={"ID":"9c7b5a24-e9ce-4270-830a-b1a4882df402","Type":"ContainerStarted","Data":"8ac2cb7cc400b22f512622704144ac80944e57d6eaf90e29564e17b31203d7e2"} Sep 16 18:41:07.223055 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.223033 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/0.log" Sep 16 18:41:07.223156 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.223074 2566 generic.go:356] "Generic (PLEG): container finished" podID="be5b5e3e-2d54-4755-bf1e-307ebaa84ad3" containerID="0abafb4181ab02d6b2399d11efbb51ccb9341f940cba503a366feb0cd783d66d" exitCode=255 Sep 16 18:41:07.223288 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.223263 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" event={"ID":"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3","Type":"ContainerDied","Data":"0abafb4181ab02d6b2399d11efbb51ccb9341f940cba503a366feb0cd783d66d"} Sep 16 18:41:07.223427 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.223413 2566 scope.go:117] "RemoveContainer" containerID="0abafb4181ab02d6b2399d11efbb51ccb9341f940cba503a366feb0cd783d66d" Sep 16 18:41:07.224912 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.224887 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-dz57d" event={"ID":"3ff4c1db-9b70-4dae-977e-85493cace17a","Type":"ContainerStarted","Data":"90db2815262b5ad045f67b45c7902a0a9c2e043d40614c39772b1f678f350d9a"} Sep 16 18:41:07.227167 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.227147 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" event={"ID":"27de7a26-8b7b-4003-8b4e-83232c28c32d","Type":"ContainerStarted","Data":"a2bd2cfe87b3261c0714fe9efb259ad53bbc2158ad541277e553a99725e08b57"} Sep 16 18:41:07.228736 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.228716 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" event={"ID":"836dca5e-9f3b-4d19-9086-6453deee99e4","Type":"ContainerStarted","Data":"baef842d4e29f4ff6ef1b06cf54276119cdb7ad305aa7aa31c50159257fa3e3d"} Sep 16 18:41:07.250924 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.250888 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-qn2k4" podStartSLOduration=1.939393186 podStartE2EDuration="8.250876142s" totalImagesPullingTime="6.311482956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:07.250236166 +0000 UTC m=+52.931090777" watchObservedRunningTime="2026-09-16 18:41:07.250876142 +0000 UTC m=+52.931730773" Sep 16 18:41:07.303299 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.303255 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" podStartSLOduration=2.02362449 podStartE2EDuration="8.303237674s" totalImagesPullingTime="6.279613184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:07.301476746 +0000 UTC m=+52.982331381" watchObservedRunningTime="2026-09-16 18:41:07.303237674 +0000 UTC m=+52.984092308" Sep 16 18:41:07.328296 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.328251 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" podStartSLOduration=2.077482625 podStartE2EDuration="8.328233414s" totalImagesPullingTime="6.250750789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:07.326383897 +0000 UTC m=+53.007238522" watchObservedRunningTime="2026-09-16 18:41:07.328233414 +0000 UTC m=+53.009088056" Sep 16 18:41:07.394511 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.394468 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-jtvst" podStartSLOduration=2.339533832 podStartE2EDuration="8.394453316s" totalImagesPullingTime="6.054919484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:07.393314059 +0000 UTC m=+53.074168690" watchObservedRunningTime="2026-09-16 18:41:07.394453316 +0000 UTC m=+53.075307949" Sep 16 18:41:07.435727 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.435679 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-dz57d" podStartSLOduration=2.406989788 podStartE2EDuration="8.4356644s" totalImagesPullingTime="6.028674612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:07.434352525 +0000 UTC m=+53.115207158" watchObservedRunningTime="2026-09-16 18:41:07.4356644 +0000 UTC m=+53.116519032" Sep 16 18:41:07.477411 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.477350 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:41:07.477499 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:07.477461 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 16 18:41:07.477550 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:07.477533 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls podName:02248648-cbdf-4352-b6f7-c91ecd16ab6a nodeName:}" failed. No retries permitted until 2026-09-16 18:41:15.477513765 +0000 UTC m=+61.158368390 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-sw7rj" (UID: "02248648-cbdf-4352-b6f7-c91ecd16ab6a") : secret "cluster-monitoring-operator-tls" not found Sep 16 18:41:07.578319 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.578285 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:07.578462 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.578408 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-zngz8\" (UID: \"e4e71011-5834-4c22-a24b-8d0ff2e90359\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:41:07.578462 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:07.578443 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 16 18:41:07.578572 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:07.578510 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs podName:40180743-5519-4b63-8b3b-e790ef62c2be nodeName:}" failed. No retries permitted until 2026-09-16 18:41:15.578491322 +0000 UTC m=+61.259345940 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs") pod "router-default-54bb988d4b-wn9jr" (UID: "40180743-5519-4b63-8b3b-e790ef62c2be") : secret "router-metrics-certs-default" not found Sep 16 18:41:07.578572 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:07.578535 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:07.578682 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:07.578572 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 16 18:41:07.578682 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:07.578624 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls podName:e4e71011-5834-4c22-a24b-8d0ff2e90359 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:15.578607055 +0000 UTC m=+61.259461673 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-zngz8" (UID: "e4e71011-5834-4c22-a24b-8d0ff2e90359") : secret "samples-operator-tls" not found Sep 16 18:41:07.578682 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:07.578644 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle podName:40180743-5519-4b63-8b3b-e790ef62c2be nodeName:}" failed. No retries permitted until 2026-09-16 18:41:15.578634946 +0000 UTC m=+61.259489556 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle") pod "router-default-54bb988d4b-wn9jr" (UID: "40180743-5519-4b63-8b3b-e790ef62c2be") : configmap references non-existent config key: service-ca.crt Sep 16 18:41:08.235361 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.235331 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 18:41:08.235814 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.235770 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/0.log" Sep 16 18:41:08.235814 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.235803 2566 generic.go:356] "Generic (PLEG): container finished" podID="be5b5e3e-2d54-4755-bf1e-307ebaa84ad3" containerID="e2eeafbb1e04c8fe51e4e8735a2d5286a7bfdc152818caa995073e963a5cd41b" exitCode=255 Sep 16 18:41:08.235929 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.235896 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" event={"ID":"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3","Type":"ContainerDied","Data":"e2eeafbb1e04c8fe51e4e8735a2d5286a7bfdc152818caa995073e963a5cd41b"} Sep 16 18:41:08.236813 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.236791 2566 scope.go:117] "RemoveContainer" containerID="0abafb4181ab02d6b2399d11efbb51ccb9341f940cba503a366feb0cd783d66d" Sep 16 18:41:08.236923 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.236903 2566 scope.go:117] "RemoveContainer" containerID="e2eeafbb1e04c8fe51e4e8735a2d5286a7bfdc152818caa995073e963a5cd41b" Sep 16 18:41:08.237115 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:08.237084 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-n92m9_openshift-console-operator(be5b5e3e-2d54-4755-bf1e-307ebaa84ad3)\"" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" podUID="be5b5e3e-2d54-4755-bf1e-307ebaa84ad3" Sep 16 18:41:08.470553 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.470521 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5j2xj"] Sep 16 18:41:08.473351 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.473332 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.480297 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.480276 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d8hzw\"" Sep 16 18:41:08.480533 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.480518 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 16 18:41:08.480614 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.480584 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 16 18:41:08.485502 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.485459 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-crio-socket\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.485577 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.485507 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.485577 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.485531 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkn47\" (UniqueName: \"kubernetes.io/projected/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-api-access-rkn47\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.485577 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.485567 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-data-volume\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.485670 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.485606 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.511022 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.510999 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5j2xj"] Sep 16 18:41:08.586274 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.586247 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.586470 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.586276 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkn47\" (UniqueName: \"kubernetes.io/projected/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-api-access-rkn47\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.586470 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.586309 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-data-volume\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.586470 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.586341 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.586470 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:08.586360 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 16 18:41:08.586470 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:08.586416 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm podName:dbf9c1a5-94c1-40f6-8788-ce631c4f2200 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:09.086402737 +0000 UTC m=+54.767257364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5j2xj" (UID: "dbf9c1a5-94c1-40f6-8788-ce631c4f2200") : configmap "kube-rbac-proxy" not found Sep 16 18:41:08.586470 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:08.586421 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 16 18:41:08.586470 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:08.586474 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls podName:dbf9c1a5-94c1-40f6-8788-ce631c4f2200 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:09.08645943 +0000 UTC m=+54.767314056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5j2xj" (UID: "dbf9c1a5-94c1-40f6-8788-ce631c4f2200") : secret "insights-runtime-extractor-tls" not found Sep 16 18:41:08.586824 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.586535 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-crio-socket\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.586824 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.586675 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-data-volume\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.586824 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.586701 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-crio-socket\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:08.612586 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:08.612554 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkn47\" (UniqueName: \"kubernetes.io/projected/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-api-access-rkn47\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:09.090545 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:09.090513 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:09.090689 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:09.090571 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:09.090689 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:09.090635 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 16 18:41:09.090689 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:09.090679 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 16 18:41:09.090788 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:09.090707 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm podName:dbf9c1a5-94c1-40f6-8788-ce631c4f2200 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:10.09068691 +0000 UTC m=+55.771541524 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5j2xj" (UID: "dbf9c1a5-94c1-40f6-8788-ce631c4f2200") : configmap "kube-rbac-proxy" not found Sep 16 18:41:09.090788 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:09.090726 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls podName:dbf9c1a5-94c1-40f6-8788-ce631c4f2200 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:10.090715993 +0000 UTC m=+55.771570609 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5j2xj" (UID: "dbf9c1a5-94c1-40f6-8788-ce631c4f2200") : secret "insights-runtime-extractor-tls" not found Sep 16 18:41:09.240734 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:09.240713 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 18:41:09.241067 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:09.240993 2566 scope.go:117] "RemoveContainer" containerID="e2eeafbb1e04c8fe51e4e8735a2d5286a7bfdc152818caa995073e963a5cd41b" Sep 16 18:41:09.241134 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:09.241118 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-n92m9_openshift-console-operator(be5b5e3e-2d54-4755-bf1e-307ebaa84ad3)\"" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" podUID="be5b5e3e-2d54-4755-bf1e-307ebaa84ad3" Sep 16 18:41:09.813445 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:09.813419 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ldzq8_bd09345e-605b-4196-a5d3-1aa3e3010635/dns-node-resolver/0.log" Sep 16 18:41:09.874558 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:09.874529 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:41:09.874677 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:09.874587 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:41:10.100015 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:10.099958 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:10.100104 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:10.100056 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:10.100144 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:10.100119 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 16 18:41:10.100144 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:10.100126 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 16 18:41:10.100227 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:10.100165 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm podName:dbf9c1a5-94c1-40f6-8788-ce631c4f2200 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:12.100152615 +0000 UTC m=+57.781007225 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5j2xj" (UID: "dbf9c1a5-94c1-40f6-8788-ce631c4f2200") : configmap "kube-rbac-proxy" not found Sep 16 18:41:10.100227 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:10.100205 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls podName:dbf9c1a5-94c1-40f6-8788-ce631c4f2200 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:12.100169769 +0000 UTC m=+57.781024397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5j2xj" (UID: "dbf9c1a5-94c1-40f6-8788-ce631c4f2200") : secret "insights-runtime-extractor-tls" not found Sep 16 18:41:10.245642 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:10.245616 2566 scope.go:117] "RemoveContainer" containerID="e2eeafbb1e04c8fe51e4e8735a2d5286a7bfdc152818caa995073e963a5cd41b" Sep 16 18:41:10.246092 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:10.245853 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-n92m9_openshift-console-operator(be5b5e3e-2d54-4755-bf1e-307ebaa84ad3)\"" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" podUID="be5b5e3e-2d54-4755-bf1e-307ebaa84ad3" Sep 16 18:41:10.808754 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:10.808729 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tjc8j_58075b65-6a9b-4ef3-9776-2cf7c3a5e15a/node-ca/0.log" Sep 16 18:41:11.611247 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:11.611206 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rm9x6_49cff26f-250f-4a66-9156-4ca5778fa510/serve-healthcheck-canary/0.log" Sep 16 18:41:12.114378 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:12.114349 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:12.114558 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:12.114426 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:12.114558 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:12.114474 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 16 18:41:12.114558 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:12.114544 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm podName:dbf9c1a5-94c1-40f6-8788-ce631c4f2200 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:16.11452371 +0000 UTC m=+61.795378337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5j2xj" (UID: "dbf9c1a5-94c1-40f6-8788-ce631c4f2200") : configmap "kube-rbac-proxy" not found Sep 16 18:41:12.114786 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:12.114614 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 16 18:41:12.114786 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:12.114704 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls podName:dbf9c1a5-94c1-40f6-8788-ce631c4f2200 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:16.114683217 +0000 UTC m=+61.795537841 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5j2xj" (UID: "dbf9c1a5-94c1-40f6-8788-ce631c4f2200") : secret "insights-runtime-extractor-tls" not found Sep 16 18:41:15.131459 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:15.131430 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-b55tw" Sep 16 18:41:15.541431 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:15.541392 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:41:15.541610 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:15.541540 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 16 18:41:15.541654 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:15.541619 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls podName:02248648-cbdf-4352-b6f7-c91ecd16ab6a nodeName:}" failed. No retries permitted until 2026-09-16 18:41:31.54160333 +0000 UTC m=+77.222457939 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-sw7rj" (UID: "02248648-cbdf-4352-b6f7-c91ecd16ab6a") : secret "cluster-monitoring-operator-tls" not found Sep 16 18:41:15.641667 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:15.641637 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:15.641818 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:15.641681 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:15.641818 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:15.641801 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle podName:40180743-5519-4b63-8b3b-e790ef62c2be nodeName:}" failed. No retries permitted until 2026-09-16 18:41:31.641783618 +0000 UTC m=+77.322638232 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle") pod "router-default-54bb988d4b-wn9jr" (UID: "40180743-5519-4b63-8b3b-e790ef62c2be") : configmap references non-existent config key: service-ca.crt Sep 16 18:41:15.641918 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:15.641847 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 16 18:41:15.641918 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:15.641891 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs podName:40180743-5519-4b63-8b3b-e790ef62c2be nodeName:}" failed. No retries permitted until 2026-09-16 18:41:31.641879557 +0000 UTC m=+77.322734214 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs") pod "router-default-54bb988d4b-wn9jr" (UID: "40180743-5519-4b63-8b3b-e790ef62c2be") : secret "router-metrics-certs-default" not found Sep 16 18:41:15.641992 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:15.641950 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-zngz8\" (UID: \"e4e71011-5834-4c22-a24b-8d0ff2e90359\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:41:15.644527 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:15.644504 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e4e71011-5834-4c22-a24b-8d0ff2e90359-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-zngz8\" (UID: \"e4e71011-5834-4c22-a24b-8d0ff2e90359\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:41:15.692469 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:15.692443 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-p4q2n\"" Sep 16 18:41:15.701281 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:15.701262 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" Sep 16 18:41:15.991170 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:15.991100 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8"] Sep 16 18:41:16.145940 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:16.145917 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:16.146270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:16.146011 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:16.146270 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:16.146054 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 16 18:41:16.146270 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:16.146075 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 16 18:41:16.146270 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:16.146105 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls podName:dbf9c1a5-94c1-40f6-8788-ce631c4f2200 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:24.146090885 +0000 UTC m=+69.826945496 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5j2xj" (UID: "dbf9c1a5-94c1-40f6-8788-ce631c4f2200") : secret "insights-runtime-extractor-tls" not found Sep 16 18:41:16.146270 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:16.146121 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm podName:dbf9c1a5-94c1-40f6-8788-ce631c4f2200 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:24.146113362 +0000 UTC m=+69.826967971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5j2xj" (UID: "dbf9c1a5-94c1-40f6-8788-ce631c4f2200") : configmap "kube-rbac-proxy" not found Sep 16 18:41:16.264910 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:16.264884 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" event={"ID":"e4e71011-5834-4c22-a24b-8d0ff2e90359","Type":"ContainerStarted","Data":"ee04b8899978ed3360e8e667b4e6ced75ed475bfe61c61e159c8c336615dc42a"} Sep 16 18:41:18.272940 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:18.272909 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" event={"ID":"e4e71011-5834-4c22-a24b-8d0ff2e90359","Type":"ContainerStarted","Data":"25cad628c1814bc0b78bd367cbaa125798c68ec92646f85508dcbff094ce6735"} Sep 16 18:41:18.273353 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:18.272959 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" event={"ID":"e4e71011-5834-4c22-a24b-8d0ff2e90359","Type":"ContainerStarted","Data":"ee9103b646f50d3dddee6c8635b35476127f5c758ef502174daa0b31afa73b52"} Sep 16 18:41:18.311313 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:18.311264 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-zngz8" podStartSLOduration=17.72846386 podStartE2EDuration="19.311250011s" totalImagesPullingTime="1.582786151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:18.311224813 +0000 UTC m=+63.992079446" watchObservedRunningTime="2026-09-16 18:41:18.311250011 +0000 UTC m=+63.992104652" Sep 16 18:41:20.483081 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.483044 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:41:20.486243 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.486223 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 16 18:41:20.495765 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.495736 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/664c47f5-9768-4395-8396-ce9ac30421e3-original-pull-secret\") pod \"global-pull-secret-syncer-swmrz\" (UID: \"664c47f5-9768-4395-8396-ce9ac30421e3\") " pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:41:20.584393 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.584365 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:41:20.584503 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.584416 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:41:20.584503 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.584440 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:41:20.586526 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.586508 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46-metrics-tls\") pod \"dns-default-fvmvk\" (UID: \"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46\") " pod="openshift-dns/dns-default-fvmvk" Sep 16 18:41:20.586780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.586764 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls\") pod \"image-registry-c48d98d7b-hfww6\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:41:20.587268 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.587254 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 16 18:41:20.595129 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:20.595111 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Sep 16 18:41:20.595232 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:20.595164 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs podName:b83a34db-f76e-4f19-b377-32bba0f5fa2b nodeName:}" failed. No retries permitted until 2026-09-16 18:42:24.595147972 +0000 UTC m=+130.276002582 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs") pod "network-metrics-daemon-c2scp" (UID: "b83a34db-f76e-4f19-b377-32bba0f5fa2b") : secret "metrics-daemon-secret" not found Sep 16 18:41:20.601393 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.601377 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-swmrz" Sep 16 18:41:20.686219 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.685753 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68\") pod \"network-check-target-s4dw9\" (UID: \"3177af74-fcce-4973-9f87-3dea0506ae3d\") " pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:41:20.688590 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.688539 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/3177af74-fcce-4973-9f87-3dea0506ae3d-kube-api-access-fpp68\") pod \"network-check-target-s4dw9\" (UID: \"3177af74-fcce-4973-9f87-3dea0506ae3d\") " pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:41:20.705433 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.705413 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5qbcq\"" Sep 16 18:41:20.713743 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.713720 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:41:20.783475 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.783454 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6l2zg\"" Sep 16 18:41:20.790289 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.790248 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:41:20.813634 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.813617 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9jkmg\"" Sep 16 18:41:20.821532 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.821506 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fvmvk" Sep 16 18:41:20.909337 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:20.909311 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-swmrz"] Sep 16 18:41:20.915219 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:20.915176 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod664c47f5_9768_4395_8396_ce9ac30421e3.slice/crio-1c3b3bd4b6718cdad24f1b6f6d560b64f4a1a9b85e78a817b8245005ca3bb3d6 WatchSource:0}: Error finding container 1c3b3bd4b6718cdad24f1b6f6d560b64f4a1a9b85e78a817b8245005ca3bb3d6: Status 404 returned error can't find the container with id 1c3b3bd4b6718cdad24f1b6f6d560b64f4a1a9b85e78a817b8245005ca3bb3d6 Sep 16 18:41:21.052571 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.052542 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s4dw9"] Sep 16 18:41:21.054882 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:21.054858 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3177af74_fcce_4973_9f87_3dea0506ae3d.slice/crio-2df80a0ff7458b1e6e82cf6d40147e5f21688526d4da309c9d53044b8165a752 WatchSource:0}: Error finding container 2df80a0ff7458b1e6e82cf6d40147e5f21688526d4da309c9d53044b8165a752: Status 404 returned error can't find the container with id 2df80a0ff7458b1e6e82cf6d40147e5f21688526d4da309c9d53044b8165a752 Sep 16 18:41:21.124738 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.124718 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c48d98d7b-hfww6"] Sep 16 18:41:21.127401 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:21.127382 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e3d5a14_b8c7_4ac8_b6e3_e31f99e923e9.slice/crio-3705b5817c9841613e0d41a1a584c322b90f93af4ad03120df73f302d89b5f41 WatchSource:0}: Error finding container 3705b5817c9841613e0d41a1a584c322b90f93af4ad03120df73f302d89b5f41: Status 404 returned error can't find the container with id 3705b5817c9841613e0d41a1a584c322b90f93af4ad03120df73f302d89b5f41 Sep 16 18:41:21.140385 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.140364 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fvmvk"] Sep 16 18:41:21.144743 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:21.144723 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a6a4541_4bae_4cdd_9a65_7ad3c2b4df46.slice/crio-2781b362a9adb1e01018087e82e2cd6289f872879264757a493c981731493a38 WatchSource:0}: Error finding container 2781b362a9adb1e01018087e82e2cd6289f872879264757a493c981731493a38: Status 404 returned error can't find the container with id 2781b362a9adb1e01018087e82e2cd6289f872879264757a493c981731493a38 Sep 16 18:41:21.283440 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.283411 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s4dw9" event={"ID":"3177af74-fcce-4973-9f87-3dea0506ae3d","Type":"ContainerStarted","Data":"d80e15ed0b70674cdb4dd67bd7f7f82c3e9482260d018239ea587c2d2a899ca6"} Sep 16 18:41:21.283532 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.283463 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s4dw9" event={"ID":"3177af74-fcce-4973-9f87-3dea0506ae3d","Type":"ContainerStarted","Data":"2df80a0ff7458b1e6e82cf6d40147e5f21688526d4da309c9d53044b8165a752"} Sep 16 18:41:21.283580 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.283535 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:41:21.284698 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.284673 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-swmrz" event={"ID":"664c47f5-9768-4395-8396-ce9ac30421e3","Type":"ContainerStarted","Data":"1c3b3bd4b6718cdad24f1b6f6d560b64f4a1a9b85e78a817b8245005ca3bb3d6"} Sep 16 18:41:21.285773 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.285754 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fvmvk" event={"ID":"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46","Type":"ContainerStarted","Data":"2781b362a9adb1e01018087e82e2cd6289f872879264757a493c981731493a38"} Sep 16 18:41:21.287233 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.287210 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" event={"ID":"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9","Type":"ContainerStarted","Data":"95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44"} Sep 16 18:41:21.287320 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.287242 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" event={"ID":"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9","Type":"ContainerStarted","Data":"3705b5817c9841613e0d41a1a584c322b90f93af4ad03120df73f302d89b5f41"} Sep 16 18:41:21.287374 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.287339 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:41:21.432006 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.431913 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-s4dw9" podStartSLOduration=66.431895501 podStartE2EDuration="1m6.431895501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:21.319082937 +0000 UTC m=+66.999937569" watchObservedRunningTime="2026-09-16 18:41:21.431895501 +0000 UTC m=+67.112750134" Sep 16 18:41:21.432588 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:21.432554 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" podStartSLOduration=66.432540186 podStartE2EDuration="1m6.432540186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:21.431252515 +0000 UTC m=+67.112107150" watchObservedRunningTime="2026-09-16 18:41:21.432540186 +0000 UTC m=+67.113394818" Sep 16 18:41:22.792498 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:22.792305 2566 scope.go:117] "RemoveContainer" containerID="e2eeafbb1e04c8fe51e4e8735a2d5286a7bfdc152818caa995073e963a5cd41b" Sep 16 18:41:23.298074 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:23.298051 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 18:41:23.298232 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:23.298168 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" event={"ID":"be5b5e3e-2d54-4755-bf1e-307ebaa84ad3","Type":"ContainerStarted","Data":"2b0feec8452ebaa209eda8346d6786b00aa45ff4ccbde81918be12afad135327"} Sep 16 18:41:23.298602 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:23.298493 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:41:23.347496 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:23.347445 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" podStartSLOduration=18.125623412 podStartE2EDuration="24.347427638s" totalImagesPullingTime="6.221804226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:23.347254383 +0000 UTC m=+69.028109038" watchObservedRunningTime="2026-09-16 18:41:23.347427638 +0000 UTC m=+69.028282289" Sep 16 18:41:23.928994 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:23.928963 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-n92m9" Sep 16 18:41:24.217547 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:24.217460 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:24.217691 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:24.217565 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:24.217691 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:24.217686 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 16 18:41:24.217783 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:24.217754 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls podName:dbf9c1a5-94c1-40f6-8788-ce631c4f2200 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:40.217736032 +0000 UTC m=+85.898590655 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5j2xj" (UID: "dbf9c1a5-94c1-40f6-8788-ce631c4f2200") : secret "insights-runtime-extractor-tls" not found Sep 16 18:41:24.218001 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:24.217979 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:24.303867 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:24.303831 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fvmvk" event={"ID":"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46","Type":"ContainerStarted","Data":"a7516feb1f152eeb97c5476f3297d1f01c0ba66fa554d3684e79cbbc339ba833"} Sep 16 18:41:25.309184 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:25.309147 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fvmvk" event={"ID":"2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46","Type":"ContainerStarted","Data":"10d18c79ecb32ba1f2f9b323de4596cc52c0e93e7e87c1ad01880ae9f19fd803"} Sep 16 18:41:25.309645 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:25.309242 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fvmvk" Sep 16 18:41:25.310807 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:25.310781 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-swmrz" event={"ID":"664c47f5-9768-4395-8396-ce9ac30421e3","Type":"ContainerStarted","Data":"fb4b18b09d864e517dc53e72e7d16513917d56f0c89d46eab77d0836ca1330bd"} Sep 16 18:41:25.360101 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:25.360054 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-fvmvk" podStartSLOduration=35.313487446 podStartE2EDuration="37.360042006s" totalImagesPullingTime="2.04655456s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:25.359535891 +0000 UTC m=+71.040390522" watchObservedRunningTime="2026-09-16 18:41:25.360042006 +0000 UTC m=+71.040896637" Sep 16 18:41:25.407652 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:25.407580 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-swmrz" podStartSLOduration=66.572919279 podStartE2EDuration="1m10.407566825s" totalImagesPullingTime="3.834647546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:25.406435494 +0000 UTC m=+71.087290125" watchObservedRunningTime="2026-09-16 18:41:25.407566825 +0000 UTC m=+71.088421458" Sep 16 18:41:31.571615 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:31.571570 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:41:31.574069 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:31.574038 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/02248648-cbdf-4352-b6f7-c91ecd16ab6a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-sw7rj\" (UID: \"02248648-cbdf-4352-b6f7-c91ecd16ab6a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:41:31.672063 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:31.672035 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:31.672210 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:31.672075 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:31.672663 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:31.672633 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40180743-5519-4b63-8b3b-e790ef62c2be-service-ca-bundle\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:31.674311 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:31.674286 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40180743-5519-4b63-8b3b-e790ef62c2be-metrics-certs\") pod \"router-default-54bb988d4b-wn9jr\" (UID: \"40180743-5519-4b63-8b3b-e790ef62c2be\") " pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:31.813602 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:31.813579 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-l7txb\"" Sep 16 18:41:31.820589 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:31.820569 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" Sep 16 18:41:31.878721 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:31.878398 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g8fnt\"" Sep 16 18:41:31.878721 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:31.878633 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:32.107604 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:32.107491 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj"] Sep 16 18:41:32.109545 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:32.109522 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02248648_cbdf_4352_b6f7_c91ecd16ab6a.slice/crio-8f50cf15f5a90e8f532aa21a13b20dab8aaa07c50600808b8034f75cef3b99d2 WatchSource:0}: Error finding container 8f50cf15f5a90e8f532aa21a13b20dab8aaa07c50600808b8034f75cef3b99d2: Status 404 returned error can't find the container with id 8f50cf15f5a90e8f532aa21a13b20dab8aaa07c50600808b8034f75cef3b99d2 Sep 16 18:41:32.162350 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:32.162325 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-54bb988d4b-wn9jr"] Sep 16 18:41:32.165165 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:32.165137 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40180743_5519_4b63_8b3b_e790ef62c2be.slice/crio-0a6275aba78bedbf917e5f38c8e7ce4b11e7d158d15a7017bbe01962d1bcd5a0 WatchSource:0}: Error finding container 0a6275aba78bedbf917e5f38c8e7ce4b11e7d158d15a7017bbe01962d1bcd5a0: Status 404 returned error can't find the container with id 0a6275aba78bedbf917e5f38c8e7ce4b11e7d158d15a7017bbe01962d1bcd5a0 Sep 16 18:41:32.336676 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:32.336637 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" event={"ID":"02248648-cbdf-4352-b6f7-c91ecd16ab6a","Type":"ContainerStarted","Data":"8f50cf15f5a90e8f532aa21a13b20dab8aaa07c50600808b8034f75cef3b99d2"} Sep 16 18:41:32.338145 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:32.338116 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54bb988d4b-wn9jr" event={"ID":"40180743-5519-4b63-8b3b-e790ef62c2be","Type":"ContainerStarted","Data":"799901041ef137b547160069175d552a06a38ee0afcf555088002a4727a7f723"} Sep 16 18:41:32.338270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:32.338157 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-54bb988d4b-wn9jr" event={"ID":"40180743-5519-4b63-8b3b-e790ef62c2be","Type":"ContainerStarted","Data":"0a6275aba78bedbf917e5f38c8e7ce4b11e7d158d15a7017bbe01962d1bcd5a0"} Sep 16 18:41:32.369492 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:32.369404 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-54bb988d4b-wn9jr" podStartSLOduration=33.369384788 podStartE2EDuration="33.369384788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:32.369214121 +0000 UTC m=+78.050068774" watchObservedRunningTime="2026-09-16 18:41:32.369384788 +0000 UTC m=+78.050239421" Sep 16 18:41:32.879402 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:32.879371 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:32.882174 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:32.882148 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:33.335627 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.335597 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942"] Sep 16 18:41:33.339889 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.339869 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942" Sep 16 18:41:33.341644 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.341617 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:33.343330 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.343225 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-54bb988d4b-wn9jr" Sep 16 18:41:33.344413 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.344367 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 16 18:41:33.344523 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.344450 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 16 18:41:33.344523 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.344372 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-fc8gj\"" Sep 16 18:41:33.364257 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.363908 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942"] Sep 16 18:41:33.417969 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.417936 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-849c6cb755-l4khs"] Sep 16 18:41:33.422870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.422846 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.424658 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.424633 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-44pdw"] Sep 16 18:41:33.426867 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.426834 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 16 18:41:33.427096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.427074 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 16 18:41:33.427269 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.427248 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 16 18:41:33.427513 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.427483 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-c54mj\"" Sep 16 18:41:33.427700 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.427683 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 16 18:41:33.427948 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.427793 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 16 18:41:33.428774 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.428755 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-44pdw" Sep 16 18:41:33.428999 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.428978 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 16 18:41:33.429386 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.429367 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 16 18:41:33.437234 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.437216 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-z6sfm\"" Sep 16 18:41:33.455178 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.455123 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-849c6cb755-l4khs"] Sep 16 18:41:33.458412 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.458388 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-44pdw"] Sep 16 18:41:33.489066 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.489040 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7239f659-5512-4185-86e1-397684975a7b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-8s942\" (UID: \"7239f659-5512-4185-86e1-397684975a7b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942" Sep 16 18:41:33.489209 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.489073 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-oauth-config\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.489336 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.489208 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7239f659-5512-4185-86e1-397684975a7b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-8s942\" (UID: \"7239f659-5512-4185-86e1-397684975a7b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942" Sep 16 18:41:33.489336 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.489254 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-service-ca\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.489457 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.489345 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-oauth-serving-cert\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.489457 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.489450 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-config\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.489564 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.489478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45sl5\" (UniqueName: \"kubernetes.io/projected/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-kube-api-access-45sl5\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.489564 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.489550 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-serving-cert\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.489666 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.489605 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckvcl\" (UniqueName: \"kubernetes.io/projected/31cea14b-8e78-4949-8261-b57b97651e3b-kube-api-access-ckvcl\") pod \"downloads-76f99584f4-44pdw\" (UID: \"31cea14b-8e78-4949-8261-b57b97651e3b\") " pod="openshift-console/downloads-76f99584f4-44pdw" Sep 16 18:41:33.590270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.590177 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-service-ca\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.590270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.590243 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-oauth-serving-cert\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.590493 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.590288 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-config\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.590493 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.590318 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45sl5\" (UniqueName: \"kubernetes.io/projected/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-kube-api-access-45sl5\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.590493 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.590355 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-serving-cert\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.590493 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.590394 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckvcl\" (UniqueName: \"kubernetes.io/projected/31cea14b-8e78-4949-8261-b57b97651e3b-kube-api-access-ckvcl\") pod \"downloads-76f99584f4-44pdw\" (UID: \"31cea14b-8e78-4949-8261-b57b97651e3b\") " pod="openshift-console/downloads-76f99584f4-44pdw" Sep 16 18:41:33.590493 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.590450 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7239f659-5512-4185-86e1-397684975a7b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-8s942\" (UID: \"7239f659-5512-4185-86e1-397684975a7b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942" Sep 16 18:41:33.590493 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.590475 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-oauth-config\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.590779 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.590668 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7239f659-5512-4185-86e1-397684975a7b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-8s942\" (UID: \"7239f659-5512-4185-86e1-397684975a7b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942" Sep 16 18:41:33.591067 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.590976 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-service-ca\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.591356 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.591292 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7239f659-5512-4185-86e1-397684975a7b-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-8s942\" (UID: \"7239f659-5512-4185-86e1-397684975a7b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942" Sep 16 18:41:33.591356 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.591318 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-config\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.591702 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.591646 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-oauth-serving-cert\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.593157 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.593134 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-oauth-config\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.593294 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.593274 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-serving-cert\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.593391 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.593374 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7239f659-5512-4185-86e1-397684975a7b-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-8s942\" (UID: \"7239f659-5512-4185-86e1-397684975a7b\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942" Sep 16 18:41:33.653677 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.653598 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942" Sep 16 18:41:33.666992 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.666966 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45sl5\" (UniqueName: \"kubernetes.io/projected/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-kube-api-access-45sl5\") pod \"console-849c6cb755-l4khs\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.711107 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.711074 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckvcl\" (UniqueName: \"kubernetes.io/projected/31cea14b-8e78-4949-8261-b57b97651e3b-kube-api-access-ckvcl\") pod \"downloads-76f99584f4-44pdw\" (UID: \"31cea14b-8e78-4949-8261-b57b97651e3b\") " pod="openshift-console/downloads-76f99584f4-44pdw" Sep 16 18:41:33.735995 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.735967 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:33.742368 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:33.742343 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-44pdw" Sep 16 18:41:34.106806 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:34.106781 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942"] Sep 16 18:41:34.109752 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:34.109723 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7239f659_5512_4185_86e1_397684975a7b.slice/crio-9d0604a5b137aa384c1c9c51689e117cc5e42a0fd2247d14e8dffafc78446c7f WatchSource:0}: Error finding container 9d0604a5b137aa384c1c9c51689e117cc5e42a0fd2247d14e8dffafc78446c7f: Status 404 returned error can't find the container with id 9d0604a5b137aa384c1c9c51689e117cc5e42a0fd2247d14e8dffafc78446c7f Sep 16 18:41:34.123034 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:34.123009 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-44pdw"] Sep 16 18:41:34.128573 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:34.125564 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-849c6cb755-l4khs"] Sep 16 18:41:34.347258 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:34.347220 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" event={"ID":"02248648-cbdf-4352-b6f7-c91ecd16ab6a","Type":"ContainerStarted","Data":"43ffc3c05d70211e94d9e6d97c70bc8f0eb9786e1028dd808e02d87defad11b4"} Sep 16 18:41:34.348727 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:34.348653 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942" event={"ID":"7239f659-5512-4185-86e1-397684975a7b","Type":"ContainerStarted","Data":"9d0604a5b137aa384c1c9c51689e117cc5e42a0fd2247d14e8dffafc78446c7f"} Sep 16 18:41:34.350091 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:34.350059 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-44pdw" event={"ID":"31cea14b-8e78-4949-8261-b57b97651e3b","Type":"ContainerStarted","Data":"9bad0fdf15d3724ab966202c646c5c3778e43a9d382d538fd0e76930d8fc3ca2"} Sep 16 18:41:34.351419 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:34.351393 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-849c6cb755-l4khs" event={"ID":"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e","Type":"ContainerStarted","Data":"2d784e3df387d75ee8fcdf2c393d68c105bf3d040b22250cf031de640a9f11f1"} Sep 16 18:41:34.414574 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:34.414479 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-sw7rj" podStartSLOduration=33.651028388 podStartE2EDuration="35.414463906s" totalImagesPullingTime="1.763435518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:34.413994809 +0000 UTC m=+80.094849440" watchObservedRunningTime="2026-09-16 18:41:34.414463906 +0000 UTC m=+80.095318538" Sep 16 18:41:35.316793 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:35.316761 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fvmvk" Sep 16 18:41:36.363399 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:36.363363 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942" event={"ID":"7239f659-5512-4185-86e1-397684975a7b","Type":"ContainerStarted","Data":"ace20fcf8709abe28d11aa88360b4f49e476343615129d028119ae12fddb1fea"} Sep 16 18:41:36.430166 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:36.430105 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8s942" podStartSLOduration=1.814966794 podStartE2EDuration="3.430088233s" totalImagesPullingTime="1.615121439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:36.428955687 +0000 UTC m=+82.109810320" watchObservedRunningTime="2026-09-16 18:41:36.430088233 +0000 UTC m=+82.110942894" Sep 16 18:41:38.372140 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:38.372099 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-849c6cb755-l4khs" event={"ID":"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e","Type":"ContainerStarted","Data":"b492c222624c13a6ba1c87bd7b8e7804e8ce34f15c05663e661af6f2d10998d7"} Sep 16 18:41:38.424130 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:38.424082 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-849c6cb755-l4khs" podStartSLOduration=1.853660893 podStartE2EDuration="5.424067356s" totalImagesPullingTime="3.570406463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:38.423885158 +0000 UTC m=+84.104739791" watchObservedRunningTime="2026-09-16 18:41:38.424067356 +0000 UTC m=+84.104922000" Sep 16 18:41:40.249454 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:40.249412 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:40.252034 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:40.252010 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/dbf9c1a5-94c1-40f6-8788-ce631c4f2200-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5j2xj\" (UID: \"dbf9c1a5-94c1-40f6-8788-ce631c4f2200\") " pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:40.284815 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:40.284788 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d8hzw\"" Sep 16 18:41:40.292257 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:40.292236 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5j2xj" Sep 16 18:41:40.593450 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:40.593283 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5j2xj"] Sep 16 18:41:40.596779 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:40.596743 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf9c1a5_94c1_40f6_8788_ce631c4f2200.slice/crio-fef363c479353e55266c6ece73ea9d582d0ddd4c46372985c25f7b7a4e3d5945 WatchSource:0}: Error finding container fef363c479353e55266c6ece73ea9d582d0ddd4c46372985c25f7b7a4e3d5945: Status 404 returned error can't find the container with id fef363c479353e55266c6ece73ea9d582d0ddd4c46372985c25f7b7a4e3d5945 Sep 16 18:41:40.795847 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:40.795810 2566 patch_prober.go:28] interesting pod/image-registry-c48d98d7b-hfww6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 16 18:41:40.796005 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:40.795870 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" podUID="6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:41:41.391261 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:41.391221 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5j2xj" event={"ID":"dbf9c1a5-94c1-40f6-8788-ce631c4f2200","Type":"ContainerStarted","Data":"21d6f8f94ee5e71d4d18d381e4c1ce77ba20cb10b235697654b2fa604a48b94d"} Sep 16 18:41:41.391583 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:41.391308 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5j2xj" event={"ID":"dbf9c1a5-94c1-40f6-8788-ce631c4f2200","Type":"ContainerStarted","Data":"24859cd3fbfb72c41253897a4ff497824348dae2cf32497ac09e049f0e0d78b1"} Sep 16 18:41:41.391583 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:41.391330 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5j2xj" event={"ID":"dbf9c1a5-94c1-40f6-8788-ce631c4f2200","Type":"ContainerStarted","Data":"fef363c479353e55266c6ece73ea9d582d0ddd4c46372985c25f7b7a4e3d5945"} Sep 16 18:41:42.298446 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:42.298415 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:41:43.406904 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:43.406871 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5j2xj" event={"ID":"dbf9c1a5-94c1-40f6-8788-ce631c4f2200","Type":"ContainerStarted","Data":"f96416af2068ced7b97b8aefdd748730430ebe133b3099187e28f83b02c34664"} Sep 16 18:41:43.438276 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:43.438225 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5j2xj" podStartSLOduration=33.427472828 podStartE2EDuration="35.438183113s" totalImagesPullingTime="2.010710285s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 18:41:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:43.436914166 +0000 UTC m=+89.117768798" watchObservedRunningTime="2026-09-16 18:41:43.438183113 +0000 UTC m=+89.119037747" Sep 16 18:41:43.736369 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:43.736262 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:43.736369 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:43.736332 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:43.742028 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:43.742001 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:44.416109 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:44.416080 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:41:45.151133 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.151097 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-np77d"] Sep 16 18:41:45.157078 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.157054 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.162278 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.162257 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 16 18:41:45.162410 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.162308 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 16 18:41:45.163942 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.163921 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nr5ld\"" Sep 16 18:41:45.164054 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.163921 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 16 18:41:45.165025 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.165007 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 16 18:41:45.296612 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.296578 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-sys\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.296786 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.296617 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-textfile\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.296786 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.296647 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-accelerators-collector-config\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.296786 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.296669 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7lk2\" (UniqueName: \"kubernetes.io/projected/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-kube-api-access-k7lk2\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.296786 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.296711 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-tls\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.296786 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.296758 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-metrics-client-ca\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.296786 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.296776 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-wtmp\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.297085 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.296827 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-root\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.297085 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.296878 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.397994 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.397962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-tls\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.398151 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.398008 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-metrics-client-ca\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.398151 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.398025 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-wtmp\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.398151 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.398076 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-root\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.398151 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.398102 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.398151 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.398119 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-sys\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.398151 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.398133 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-textfile\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.398151 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:45.398145 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 16 18:41:45.398648 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:41:45.398257 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-tls podName:e28b89a4-0e37-40a1-95ef-ee3c9bb1e549 nodeName:}" failed. No retries permitted until 2026-09-16 18:41:45.898234502 +0000 UTC m=+91.579089137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-tls") pod "node-exporter-np77d" (UID: "e28b89a4-0e37-40a1-95ef-ee3c9bb1e549") : secret "node-exporter-tls" not found Sep 16 18:41:45.398648 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.398149 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-accelerators-collector-config\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.398648 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.398532 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7lk2\" (UniqueName: \"kubernetes.io/projected/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-kube-api-access-k7lk2\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.398803 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.398665 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-accelerators-collector-config\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.399022 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.398917 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-root\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.399022 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.398970 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-sys\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.399022 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.398990 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-wtmp\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.399224 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.399115 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-metrics-client-ca\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.399290 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.399235 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-textfile\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.401384 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.401339 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.432731 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.432711 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7lk2\" (UniqueName: \"kubernetes.io/projected/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-kube-api-access-k7lk2\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.903036 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.902998 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-tls\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:45.905871 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:45.905813 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e28b89a4-0e37-40a1-95ef-ee3c9bb1e549-node-exporter-tls\") pod \"node-exporter-np77d\" (UID: \"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549\") " pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:46.067270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:46.067229 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-np77d" Sep 16 18:41:51.056873 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:41:51.056834 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode28b89a4_0e37_40a1_95ef_ee3c9bb1e549.slice/crio-70ee2c2b93b950ee760a12b51e436f114c9354d99b6da596344cd8a8343c08bf WatchSource:0}: Error finding container 70ee2c2b93b950ee760a12b51e436f114c9354d99b6da596344cd8a8343c08bf: Status 404 returned error can't find the container with id 70ee2c2b93b950ee760a12b51e436f114c9354d99b6da596344cd8a8343c08bf Sep 16 18:41:51.444645 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:51.444559 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-np77d" event={"ID":"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549","Type":"ContainerStarted","Data":"70ee2c2b93b950ee760a12b51e436f114c9354d99b6da596344cd8a8343c08bf"} Sep 16 18:41:51.446717 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:51.446673 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-44pdw" event={"ID":"31cea14b-8e78-4949-8261-b57b97651e3b","Type":"ContainerStarted","Data":"ff318612309b52c40eb5d49c22a94631db86095d250c72d3038a6bd815f15351"} Sep 16 18:41:51.446914 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:51.446887 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-44pdw" Sep 16 18:41:51.466726 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:51.466701 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-44pdw" Sep 16 18:41:51.474465 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:51.474416 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-44pdw" podStartSLOduration=1.454923041 podStartE2EDuration="18.474396899s" totalImagesPullingTime="17.019473858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:41:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:51.473477637 +0000 UTC m=+97.154332271" watchObservedRunningTime="2026-09-16 18:41:51.474396899 +0000 UTC m=+97.155251532" Sep 16 18:41:52.296915 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:52.296881 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-s4dw9" Sep 16 18:41:52.453293 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:52.453208 2566 generic.go:356] "Generic (PLEG): container finished" podID="e28b89a4-0e37-40a1-95ef-ee3c9bb1e549" containerID="8686c31505ba682bd76c0788ca88035a6a06fabe8b7060a2f4d8c231be5bb42a" exitCode=0 Sep 16 18:41:52.453442 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:52.453288 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-np77d" event={"ID":"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549","Type":"ContainerDied","Data":"8686c31505ba682bd76c0788ca88035a6a06fabe8b7060a2f4d8c231be5bb42a"} Sep 16 18:41:53.460286 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:53.460235 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-np77d" event={"ID":"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549","Type":"ContainerStarted","Data":"500e95e292fc41909b3bb316531196264e371cfba220cf9943014f0e7cb7368a"} Sep 16 18:41:53.460736 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:53.460312 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-np77d" event={"ID":"e28b89a4-0e37-40a1-95ef-ee3c9bb1e549","Type":"ContainerStarted","Data":"5c5bd95d09494adacfed6d8aa3cc42a0280b1361e48493356c400fd5297a0c4b"} Sep 16 18:41:53.529679 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:53.529621 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-np77d" podStartSLOduration=7.352826486 podStartE2EDuration="8.529598959s" totalImagesPullingTime="1.101969748s" totalInitContainerRuntime="74.802725ms" isStatefulPod=true podCreationTimestamp="2026-09-16 18:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:41:53.527890261 +0000 UTC m=+99.208744905" watchObservedRunningTime="2026-09-16 18:41:53.529598959 +0000 UTC m=+99.210453592" Sep 16 18:41:55.639373 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:41:55.639329 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c48d98d7b-hfww6"] Sep 16 18:42:06.369610 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:06.369573 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-849c6cb755-l4khs"] Sep 16 18:42:20.662079 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.662019 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" podUID="6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" containerName="registry" containerID="cri-o://95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44" gracePeriod=30 Sep 16 18:42:20.924086 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.924035 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:42:20.971292 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.971264 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-trusted-ca\") pod \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " Sep 16 18:42:20.971414 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.971330 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-certificates\") pod \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " Sep 16 18:42:20.971414 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.971351 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-installation-pull-secrets\") pod \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " Sep 16 18:42:20.971414 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.971372 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-ca-trust-extracted\") pod \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " Sep 16 18:42:20.971414 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.971402 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-bound-sa-token\") pod \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " Sep 16 18:42:20.971617 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.971423 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-image-registry-private-configuration\") pod \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " Sep 16 18:42:20.971617 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.971450 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mljrv\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-kube-api-access-mljrv\") pod \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " Sep 16 18:42:20.971617 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.971492 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls\") pod \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\" (UID: \"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9\") " Sep 16 18:42:20.971768 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.971678 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 18:42:20.971835 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.971761 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 18:42:20.973870 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.973830 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 18:42:20.973978 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.973925 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 18:42:20.974414 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.974379 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 18:42:20.974511 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.974432 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 18:42:20.974552 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.974524 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-kube-api-access-mljrv" (OuterVolumeSpecName: "kube-api-access-mljrv") pod "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9"). InnerVolumeSpecName "kube-api-access-mljrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 18:42:20.982863 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:20.982831 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" (UID: "6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 18:42:21.072466 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.072436 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-certificates\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:21.072466 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.072460 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-installation-pull-secrets\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:21.072466 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.072470 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-ca-trust-extracted\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:21.072657 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.072479 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-bound-sa-token\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:21.072657 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.072488 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-image-registry-private-configuration\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:21.072657 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.072497 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mljrv\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-kube-api-access-mljrv\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:21.072657 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.072505 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-registry-tls\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:21.072657 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.072513 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9-trusted-ca\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:21.572179 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.572142 2566 generic.go:356] "Generic (PLEG): container finished" podID="6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" containerID="95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44" exitCode=0 Sep 16 18:42:21.572359 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.572200 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" Sep 16 18:42:21.572359 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.572224 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" event={"ID":"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9","Type":"ContainerDied","Data":"95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44"} Sep 16 18:42:21.572359 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.572276 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c48d98d7b-hfww6" event={"ID":"6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9","Type":"ContainerDied","Data":"3705b5817c9841613e0d41a1a584c322b90f93af4ad03120df73f302d89b5f41"} Sep 16 18:42:21.572359 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.572297 2566 scope.go:117] "RemoveContainer" containerID="95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44" Sep 16 18:42:21.584611 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.584594 2566 scope.go:117] "RemoveContainer" containerID="95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44" Sep 16 18:42:21.584900 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:42:21.584880 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44\": container with ID starting with 95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44 not found: ID does not exist" containerID="95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44" Sep 16 18:42:21.584958 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.584909 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44"} err="failed to get container status \"95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44\": rpc error: code = NotFound desc = could not find container \"95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44\": container with ID starting with 95200fa919e1dba6358ce7b41277e26e35f059ba948d702781b56d02df27bf44 not found: ID does not exist" Sep 16 18:42:21.602716 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.602692 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c48d98d7b-hfww6"] Sep 16 18:42:21.606916 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:21.606896 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-c48d98d7b-hfww6"] Sep 16 18:42:22.796586 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:22.796554 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" path="/var/lib/kubelet/pods/6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9/volumes" Sep 16 18:42:24.603620 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:24.603582 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:42:24.605919 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:24.605894 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b83a34db-f76e-4f19-b377-32bba0f5fa2b-metrics-certs\") pod \"network-metrics-daemon-c2scp\" (UID: \"b83a34db-f76e-4f19-b377-32bba0f5fa2b\") " pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:42:24.809200 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:24.809168 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ndh2n\"" Sep 16 18:42:24.817256 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:24.817238 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c2scp" Sep 16 18:42:25.138332 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:25.138307 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c2scp"] Sep 16 18:42:25.140332 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:42:25.140305 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb83a34db_f76e_4f19_b377_32bba0f5fa2b.slice/crio-b2c371b97578c3a5f3f0beee92ae8129ff713371afe2e0e52adf5da2eda38e28 WatchSource:0}: Error finding container b2c371b97578c3a5f3f0beee92ae8129ff713371afe2e0e52adf5da2eda38e28: Status 404 returned error can't find the container with id b2c371b97578c3a5f3f0beee92ae8129ff713371afe2e0e52adf5da2eda38e28 Sep 16 18:42:25.591178 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:25.591135 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c2scp" event={"ID":"b83a34db-f76e-4f19-b377-32bba0f5fa2b","Type":"ContainerStarted","Data":"b2c371b97578c3a5f3f0beee92ae8129ff713371afe2e0e52adf5da2eda38e28"} Sep 16 18:42:26.597204 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:26.597157 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c2scp" event={"ID":"b83a34db-f76e-4f19-b377-32bba0f5fa2b","Type":"ContainerStarted","Data":"11092d5247e4572bf49ee9f04c72c030333f54f61e1eecc523686a529dd484a4"} Sep 16 18:42:27.602137 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:27.602045 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c2scp" event={"ID":"b83a34db-f76e-4f19-b377-32bba0f5fa2b","Type":"ContainerStarted","Data":"884808ce601e1db946d2088cbb785e3c2788bd9294f3e540fcb23669cae87561"} Sep 16 18:42:27.603637 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:27.603610 2566 generic.go:356] "Generic (PLEG): container finished" podID="3ff4c1db-9b70-4dae-977e-85493cace17a" containerID="90db2815262b5ad045f67b45c7902a0a9c2e043d40614c39772b1f678f350d9a" exitCode=0 Sep 16 18:42:27.603762 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:27.603688 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-dz57d" event={"ID":"3ff4c1db-9b70-4dae-977e-85493cace17a","Type":"ContainerDied","Data":"90db2815262b5ad045f67b45c7902a0a9c2e043d40614c39772b1f678f350d9a"} Sep 16 18:42:27.604027 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:27.604009 2566 scope.go:117] "RemoveContainer" containerID="90db2815262b5ad045f67b45c7902a0a9c2e043d40614c39772b1f678f350d9a" Sep 16 18:42:27.605137 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:27.605118 2566 generic.go:356] "Generic (PLEG): container finished" podID="836dca5e-9f3b-4d19-9086-6453deee99e4" containerID="baef842d4e29f4ff6ef1b06cf54276119cdb7ad305aa7aa31c50159257fa3e3d" exitCode=0 Sep 16 18:42:27.605233 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:27.605149 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" event={"ID":"836dca5e-9f3b-4d19-9086-6453deee99e4","Type":"ContainerDied","Data":"baef842d4e29f4ff6ef1b06cf54276119cdb7ad305aa7aa31c50159257fa3e3d"} Sep 16 18:42:27.605400 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:27.605388 2566 scope.go:117] "RemoveContainer" containerID="baef842d4e29f4ff6ef1b06cf54276119cdb7ad305aa7aa31c50159257fa3e3d" Sep 16 18:42:27.625021 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:27.624985 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-c2scp" podStartSLOduration=131.326028547 podStartE2EDuration="2m12.62497283s" totalImagesPullingTime="1.298944283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:40:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:42:27.623088095 +0000 UTC m=+133.303942728" watchObservedRunningTime="2026-09-16 18:42:27.62497283 +0000 UTC m=+133.305827462" Sep 16 18:42:28.611178 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:28.611144 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-nvnqb" event={"ID":"836dca5e-9f3b-4d19-9086-6453deee99e4","Type":"ContainerStarted","Data":"423cc3443c492538f400435bb85bb0d49de9e76c0229ed62e0a73ba0b8f50607"} Sep 16 18:42:28.613206 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:28.613156 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-dz57d" event={"ID":"3ff4c1db-9b70-4dae-977e-85493cace17a","Type":"ContainerStarted","Data":"cf10553c2dbbcae3cb729001c92d7b484f49e7e868746ee039bc9c00e01c3d49"} Sep 16 18:42:31.389299 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.389230 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-849c6cb755-l4khs" podUID="ddff7d8e-fdc8-4f50-a484-a2167cbeea0e" containerName="console" containerID="cri-o://b492c222624c13a6ba1c87bd7b8e7804e8ce34f15c05663e661af6f2d10998d7" gracePeriod=15 Sep 16 18:42:31.628496 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.628469 2566 generic.go:356] "Generic (PLEG): container finished" podID="ddff7d8e-fdc8-4f50-a484-a2167cbeea0e" containerID="b492c222624c13a6ba1c87bd7b8e7804e8ce34f15c05663e661af6f2d10998d7" exitCode=0 Sep 16 18:42:31.628625 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.628514 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-849c6cb755-l4khs" event={"ID":"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e","Type":"ContainerDied","Data":"b492c222624c13a6ba1c87bd7b8e7804e8ce34f15c05663e661af6f2d10998d7"} Sep 16 18:42:31.628625 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.628559 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-849c6cb755-l4khs" event={"ID":"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e","Type":"ContainerDied","Data":"2d784e3df387d75ee8fcdf2c393d68c105bf3d040b22250cf031de640a9f11f1"} Sep 16 18:42:31.628625 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.628580 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d784e3df387d75ee8fcdf2c393d68c105bf3d040b22250cf031de640a9f11f1" Sep 16 18:42:31.660678 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.660629 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:42:31.760142 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.760105 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-oauth-serving-cert\") pod \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " Sep 16 18:42:31.760288 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.760178 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-serving-cert\") pod \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " Sep 16 18:42:31.760288 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.760219 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-oauth-config\") pod \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " Sep 16 18:42:31.760288 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.760243 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-service-ca\") pod \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " Sep 16 18:42:31.760288 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.760268 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-config\") pod \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " Sep 16 18:42:31.760503 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.760300 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45sl5\" (UniqueName: \"kubernetes.io/projected/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-kube-api-access-45sl5\") pod \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\" (UID: \"ddff7d8e-fdc8-4f50-a484-a2167cbeea0e\") " Sep 16 18:42:31.760673 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.760639 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ddff7d8e-fdc8-4f50-a484-a2167cbeea0e" (UID: "ddff7d8e-fdc8-4f50-a484-a2167cbeea0e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 18:42:31.760790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.760760 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-service-ca" (OuterVolumeSpecName: "service-ca") pod "ddff7d8e-fdc8-4f50-a484-a2167cbeea0e" (UID: "ddff7d8e-fdc8-4f50-a484-a2167cbeea0e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 18:42:31.760841 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.760776 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-config" (OuterVolumeSpecName: "console-config") pod "ddff7d8e-fdc8-4f50-a484-a2167cbeea0e" (UID: "ddff7d8e-fdc8-4f50-a484-a2167cbeea0e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 18:42:31.762471 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.762444 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ddff7d8e-fdc8-4f50-a484-a2167cbeea0e" (UID: "ddff7d8e-fdc8-4f50-a484-a2167cbeea0e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 18:42:31.762531 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.762496 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ddff7d8e-fdc8-4f50-a484-a2167cbeea0e" (UID: "ddff7d8e-fdc8-4f50-a484-a2167cbeea0e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 18:42:31.762567 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.762551 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-kube-api-access-45sl5" (OuterVolumeSpecName: "kube-api-access-45sl5") pod "ddff7d8e-fdc8-4f50-a484-a2167cbeea0e" (UID: "ddff7d8e-fdc8-4f50-a484-a2167cbeea0e"). InnerVolumeSpecName "kube-api-access-45sl5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 18:42:31.861270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.861248 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-service-ca\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:31.861270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.861268 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-config\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:31.861388 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.861302 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45sl5\" (UniqueName: \"kubernetes.io/projected/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-kube-api-access-45sl5\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:31.861388 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.861313 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-oauth-serving-cert\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:31.861388 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.861323 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-serving-cert\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:31.861388 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:31.861331 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e-console-oauth-config\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:42:32.632524 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:32.632487 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-849c6cb755-l4khs" Sep 16 18:42:32.657545 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:32.657521 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-849c6cb755-l4khs"] Sep 16 18:42:32.663101 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:32.663078 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-849c6cb755-l4khs"] Sep 16 18:42:32.795786 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:32.795742 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddff7d8e-fdc8-4f50-a484-a2167cbeea0e" path="/var/lib/kubelet/pods/ddff7d8e-fdc8-4f50-a484-a2167cbeea0e/volumes" Sep 16 18:42:43.691934 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:43.691885 2566 generic.go:356] "Generic (PLEG): container finished" podID="27de7a26-8b7b-4003-8b4e-83232c28c32d" containerID="a2bd2cfe87b3261c0714fe9efb259ad53bbc2158ad541277e553a99725e08b57" exitCode=0 Sep 16 18:42:43.692385 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:43.691943 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" event={"ID":"27de7a26-8b7b-4003-8b4e-83232c28c32d","Type":"ContainerDied","Data":"a2bd2cfe87b3261c0714fe9efb259ad53bbc2158ad541277e553a99725e08b57"} Sep 16 18:42:43.692385 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:43.692290 2566 scope.go:117] "RemoveContainer" containerID="a2bd2cfe87b3261c0714fe9efb259ad53bbc2158ad541277e553a99725e08b57" Sep 16 18:42:44.697955 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:42:44.697915 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-gzxqx" event={"ID":"27de7a26-8b7b-4003-8b4e-83232c28c32d","Type":"ContainerStarted","Data":"5ffb15811bea8e7349e99427a248fce220035fb1dfd9567b2d8df63b5480aa4c"} Sep 16 18:43:02.435976 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.435941 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c58f5f75-bsw97"] Sep 16 18:43:02.436434 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.436312 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddff7d8e-fdc8-4f50-a484-a2167cbeea0e" containerName="console" Sep 16 18:43:02.436434 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.436328 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddff7d8e-fdc8-4f50-a484-a2167cbeea0e" containerName="console" Sep 16 18:43:02.436434 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.436344 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" containerName="registry" Sep 16 18:43:02.436434 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.436349 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" containerName="registry" Sep 16 18:43:02.436434 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.436389 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e3d5a14-b8c7-4ac8-b6e3-e31f99e923e9" containerName="registry" Sep 16 18:43:02.436434 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.436400 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddff7d8e-fdc8-4f50-a484-a2167cbeea0e" containerName="console" Sep 16 18:43:02.442458 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.442432 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.445281 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.445257 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 16 18:43:02.445411 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.445367 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 16 18:43:02.445885 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.445866 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 16 18:43:02.446037 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.446016 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 16 18:43:02.446131 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.446024 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 16 18:43:02.446241 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.446182 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-c54mj\"" Sep 16 18:43:02.453358 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.452900 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 16 18:43:02.454638 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.454612 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c58f5f75-bsw97"] Sep 16 18:43:02.576641 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.576603 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-oauth-serving-cert\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.576833 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.576648 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-serving-cert\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.576833 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.576682 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-trusted-ca-bundle\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.576833 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.576709 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-service-ca\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.576833 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.576736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-oauth-config\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.576833 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.576756 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-config\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.576833 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.576797 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdjkw\" (UniqueName: \"kubernetes.io/projected/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-kube-api-access-sdjkw\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.678107 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.678067 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-oauth-serving-cert\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.678107 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.678112 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-serving-cert\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.678360 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.678131 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-trusted-ca-bundle\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.678360 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.678151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-service-ca\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.678360 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.678208 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-oauth-config\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.678360 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.678233 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-config\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.678360 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.678261 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdjkw\" (UniqueName: \"kubernetes.io/projected/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-kube-api-access-sdjkw\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.678942 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.678907 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-service-ca\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.679080 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.678952 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-oauth-serving-cert\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.679149 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.679073 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-config\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.679231 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.679153 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-trusted-ca-bundle\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.680761 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.680735 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-oauth-config\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.680844 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.680827 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-serving-cert\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.688455 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.688408 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdjkw\" (UniqueName: \"kubernetes.io/projected/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-kube-api-access-sdjkw\") pod \"console-5c58f5f75-bsw97\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:02.753534 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:02.753504 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:03.030775 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:03.030741 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c58f5f75-bsw97"] Sep 16 18:43:03.033677 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:43:03.033654 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod201b3e0e_9b4d_45f7_b821_ad55e5239d7b.slice/crio-9963f0aa9ab907dc0064df1cb34d11b8b97a4bd812cc7a18ebf38717e58cd3d3 WatchSource:0}: Error finding container 9963f0aa9ab907dc0064df1cb34d11b8b97a4bd812cc7a18ebf38717e58cd3d3: Status 404 returned error can't find the container with id 9963f0aa9ab907dc0064df1cb34d11b8b97a4bd812cc7a18ebf38717e58cd3d3 Sep 16 18:43:03.775937 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:03.775900 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c58f5f75-bsw97" event={"ID":"201b3e0e-9b4d-45f7-b821-ad55e5239d7b","Type":"ContainerStarted","Data":"32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3"} Sep 16 18:43:03.776328 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:03.775948 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c58f5f75-bsw97" event={"ID":"201b3e0e-9b4d-45f7-b821-ad55e5239d7b","Type":"ContainerStarted","Data":"9963f0aa9ab907dc0064df1cb34d11b8b97a4bd812cc7a18ebf38717e58cd3d3"} Sep 16 18:43:12.754238 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:12.754126 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:12.754238 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:12.754206 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:12.759162 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:12.759138 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:43:12.781384 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:12.781329 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c58f5f75-bsw97" podStartSLOduration=10.781315101 podStartE2EDuration="10.781315101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:43:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:43:03.807904175 +0000 UTC m=+169.488758808" watchObservedRunningTime="2026-09-16 18:43:12.781315101 +0000 UTC m=+178.462169746" Sep 16 18:43:12.812936 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:43:12.812909 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:44:45.787321 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.787228 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb"] Sep 16 18:44:45.790612 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.790591 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:44:45.793289 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.793266 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lh8tr\"" Sep 16 18:44:45.793379 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.793283 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Sep 16 18:44:45.794034 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.794013 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Sep 16 18:44:45.803098 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.803069 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb"] Sep 16 18:44:45.898363 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.898335 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb\" (UID: \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:44:45.898464 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.898403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb\" (UID: \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:44:45.898464 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.898441 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg8nf\" (UniqueName: \"kubernetes.io/projected/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-kube-api-access-rg8nf\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb\" (UID: \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:44:45.999318 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.999274 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg8nf\" (UniqueName: \"kubernetes.io/projected/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-kube-api-access-rg8nf\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb\" (UID: \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:44:45.999318 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.999325 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb\" (UID: \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:44:45.999514 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.999374 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb\" (UID: \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:44:45.999730 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.999709 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-bundle\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb\" (UID: \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:44:45.999786 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:45.999759 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-util\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb\" (UID: \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:44:46.009661 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:46.009640 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg8nf\" (UniqueName: \"kubernetes.io/projected/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-kube-api-access-rg8nf\") pod \"1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb\" (UID: \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\") " pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:44:46.101330 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:46.101280 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:44:46.376879 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:46.376815 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb"] Sep 16 18:44:46.380787 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:44:46.380761 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a583299_bc8a_45d4_b216_7dd6e0dcbd96.slice/crio-d0126fe5ac2364c67b8fcfa074c556b8499f2c1d38a0f6ce7174a33891cd8fa8 WatchSource:0}: Error finding container d0126fe5ac2364c67b8fcfa074c556b8499f2c1d38a0f6ce7174a33891cd8fa8: Status 404 returned error can't find the container with id d0126fe5ac2364c67b8fcfa074c556b8499f2c1d38a0f6ce7174a33891cd8fa8 Sep 16 18:44:47.177749 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:47.177714 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" event={"ID":"2a583299-bc8a-45d4-b216-7dd6e0dcbd96","Type":"ContainerStarted","Data":"d0126fe5ac2364c67b8fcfa074c556b8499f2c1d38a0f6ce7174a33891cd8fa8"} Sep 16 18:44:53.204726 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:53.204687 2566 generic.go:356] "Generic (PLEG): container finished" podID="2a583299-bc8a-45d4-b216-7dd6e0dcbd96" containerID="638049420f71104fc73601c0603dd4c744fdf53015b20a346daf687469a00b3f" exitCode=0 Sep 16 18:44:53.205242 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:53.204765 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" event={"ID":"2a583299-bc8a-45d4-b216-7dd6e0dcbd96","Type":"ContainerDied","Data":"638049420f71104fc73601c0603dd4c744fdf53015b20a346daf687469a00b3f"} Sep 16 18:44:56.220499 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:56.220426 2566 generic.go:356] "Generic (PLEG): container finished" podID="2a583299-bc8a-45d4-b216-7dd6e0dcbd96" containerID="22dc5440d6c3c3f14ea59ee8743769919c295136454f50fb1372bd67bfc4d343" exitCode=0 Sep 16 18:44:56.220850 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:44:56.220522 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" event={"ID":"2a583299-bc8a-45d4-b216-7dd6e0dcbd96","Type":"ContainerDied","Data":"22dc5440d6c3c3f14ea59ee8743769919c295136454f50fb1372bd67bfc4d343"} Sep 16 18:45:03.257902 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:03.257863 2566 generic.go:356] "Generic (PLEG): container finished" podID="2a583299-bc8a-45d4-b216-7dd6e0dcbd96" containerID="20b210292fc5c155745b1656aeac949222620c69eeb3c4d0d6346047899a03ff" exitCode=0 Sep 16 18:45:03.258392 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:03.257937 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" event={"ID":"2a583299-bc8a-45d4-b216-7dd6e0dcbd96","Type":"ContainerDied","Data":"20b210292fc5c155745b1656aeac949222620c69eeb3c4d0d6346047899a03ff"} Sep 16 18:45:04.383540 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:04.383517 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:45:04.542953 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:04.542869 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rg8nf\" (UniqueName: \"kubernetes.io/projected/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-kube-api-access-rg8nf\") pod \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\" (UID: \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\") " Sep 16 18:45:04.543090 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:04.542951 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-util\") pod \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\" (UID: \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\") " Sep 16 18:45:04.543090 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:04.543015 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-bundle\") pod \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\" (UID: \"2a583299-bc8a-45d4-b216-7dd6e0dcbd96\") " Sep 16 18:45:04.543749 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:04.543721 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-bundle" (OuterVolumeSpecName: "bundle") pod "2a583299-bc8a-45d4-b216-7dd6e0dcbd96" (UID: "2a583299-bc8a-45d4-b216-7dd6e0dcbd96"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 18:45:04.545059 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:04.545030 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-kube-api-access-rg8nf" (OuterVolumeSpecName: "kube-api-access-rg8nf") pod "2a583299-bc8a-45d4-b216-7dd6e0dcbd96" (UID: "2a583299-bc8a-45d4-b216-7dd6e0dcbd96"). InnerVolumeSpecName "kube-api-access-rg8nf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 18:45:04.547252 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:04.547229 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-util" (OuterVolumeSpecName: "util") pod "2a583299-bc8a-45d4-b216-7dd6e0dcbd96" (UID: "2a583299-bc8a-45d4-b216-7dd6e0dcbd96"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 16 18:45:04.644326 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:04.644299 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-util\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:45:04.644326 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:04.644322 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:45:04.644447 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:04.644331 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rg8nf\" (UniqueName: \"kubernetes.io/projected/2a583299-bc8a-45d4-b216-7dd6e0dcbd96-kube-api-access-rg8nf\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:45:05.268043 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:05.267997 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" event={"ID":"2a583299-bc8a-45d4-b216-7dd6e0dcbd96","Type":"ContainerDied","Data":"d0126fe5ac2364c67b8fcfa074c556b8499f2c1d38a0f6ce7174a33891cd8fa8"} Sep 16 18:45:05.268043 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:05.268023 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/1acd376c2f780381bc434f5f1985f786a96c8309f7a005b52ed03f0c1ardprb" Sep 16 18:45:05.268285 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:05.268061 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0126fe5ac2364c67b8fcfa074c556b8499f2c1d38a0f6ce7174a33891cd8fa8" Sep 16 18:45:08.928197 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.928150 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q"] Sep 16 18:45:08.928558 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.928410 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a583299-bc8a-45d4-b216-7dd6e0dcbd96" containerName="pull" Sep 16 18:45:08.928558 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.928421 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a583299-bc8a-45d4-b216-7dd6e0dcbd96" containerName="pull" Sep 16 18:45:08.928558 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.928428 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a583299-bc8a-45d4-b216-7dd6e0dcbd96" containerName="extract" Sep 16 18:45:08.928558 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.928434 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a583299-bc8a-45d4-b216-7dd6e0dcbd96" containerName="extract" Sep 16 18:45:08.928558 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.928446 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a583299-bc8a-45d4-b216-7dd6e0dcbd96" containerName="util" Sep 16 18:45:08.928558 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.928451 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a583299-bc8a-45d4-b216-7dd6e0dcbd96" containerName="util" Sep 16 18:45:08.928558 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.928503 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a583299-bc8a-45d4-b216-7dd6e0dcbd96" containerName="extract" Sep 16 18:45:08.931323 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.931304 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" Sep 16 18:45:08.933979 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.933949 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Sep 16 18:45:08.933979 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.933956 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-xxjkl\"" Sep 16 18:45:08.934145 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.933955 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Sep 16 18:45:08.934145 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.934026 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Sep 16 18:45:08.944296 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:08.944274 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q"] Sep 16 18:45:09.070981 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:09.070955 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm7tx\" (UniqueName: \"kubernetes.io/projected/f00a86d3-8497-4c59-a423-6a28f499c4c0-kube-api-access-pm7tx\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-r4s2q\" (UID: \"f00a86d3-8497-4c59-a423-6a28f499c4c0\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" Sep 16 18:45:09.071077 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:09.071016 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/f00a86d3-8497-4c59-a423-6a28f499c4c0-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-r4s2q\" (UID: \"f00a86d3-8497-4c59-a423-6a28f499c4c0\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" Sep 16 18:45:09.171475 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:09.171443 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/f00a86d3-8497-4c59-a423-6a28f499c4c0-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-r4s2q\" (UID: \"f00a86d3-8497-4c59-a423-6a28f499c4c0\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" Sep 16 18:45:09.171580 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:09.171485 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm7tx\" (UniqueName: \"kubernetes.io/projected/f00a86d3-8497-4c59-a423-6a28f499c4c0-kube-api-access-pm7tx\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-r4s2q\" (UID: \"f00a86d3-8497-4c59-a423-6a28f499c4c0\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" Sep 16 18:45:09.173691 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:09.173664 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/f00a86d3-8497-4c59-a423-6a28f499c4c0-certificates\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-r4s2q\" (UID: \"f00a86d3-8497-4c59-a423-6a28f499c4c0\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" Sep 16 18:45:09.187899 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:09.187855 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm7tx\" (UniqueName: \"kubernetes.io/projected/f00a86d3-8497-4c59-a423-6a28f499c4c0-kube-api-access-pm7tx\") pod \"custom-metrics-autoscaler-operator-566c9cd69c-r4s2q\" (UID: \"f00a86d3-8497-4c59-a423-6a28f499c4c0\") " pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" Sep 16 18:45:09.241662 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:09.241640 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" Sep 16 18:45:09.516930 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:09.516892 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q"] Sep 16 18:45:09.521965 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:45:09.521937 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf00a86d3_8497_4c59_a423_6a28f499c4c0.slice/crio-767661fe216ad5b10965f8dc0e6afe2e96bf8e809e8b32a96445429e98455353 WatchSource:0}: Error finding container 767661fe216ad5b10965f8dc0e6afe2e96bf8e809e8b32a96445429e98455353: Status 404 returned error can't find the container with id 767661fe216ad5b10965f8dc0e6afe2e96bf8e809e8b32a96445429e98455353 Sep 16 18:45:10.290654 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:10.290622 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" event={"ID":"f00a86d3-8497-4c59-a423-6a28f499c4c0","Type":"ContainerStarted","Data":"767661fe216ad5b10965f8dc0e6afe2e96bf8e809e8b32a96445429e98455353"} Sep 16 18:45:14.310150 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.310115 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" event={"ID":"f00a86d3-8497-4c59-a423-6a28f499c4c0","Type":"ContainerStarted","Data":"fc43724181e0224317761497ee7009a3a3514506131cba29c9c6d22a81be05ec"} Sep 16 18:45:14.310558 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.310247 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" Sep 16 18:45:14.335622 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.335580 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" podStartSLOduration=2.230859577 podStartE2EDuration="6.33556613s" totalImagesPullingTime="4.104706553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:14.333233854 +0000 UTC m=+300.014088487" watchObservedRunningTime="2026-09-16 18:45:14.33556613 +0000 UTC m=+300.016420761" Sep 16 18:45:14.526066 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.526034 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-579c8958f7-vrcqf"] Sep 16 18:45:14.529358 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.529341 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:14.532124 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.532098 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Sep 16 18:45:14.532250 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.532140 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Sep 16 18:45:14.532372 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.532356 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-nkdmt\"" Sep 16 18:45:14.542479 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.542457 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-vrcqf"] Sep 16 18:45:14.610547 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.610482 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l8lv\" (UniqueName: \"kubernetes.io/projected/2656427d-2f81-43ab-9f1e-736e90555d46-kube-api-access-2l8lv\") pod \"keda-operator-579c8958f7-vrcqf\" (UID: \"2656427d-2f81-43ab-9f1e-736e90555d46\") " pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:14.610547 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.610522 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2656427d-2f81-43ab-9f1e-736e90555d46-certificates\") pod \"keda-operator-579c8958f7-vrcqf\" (UID: \"2656427d-2f81-43ab-9f1e-736e90555d46\") " pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:14.610680 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.610597 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/2656427d-2f81-43ab-9f1e-736e90555d46-cabundle0\") pod \"keda-operator-579c8958f7-vrcqf\" (UID: \"2656427d-2f81-43ab-9f1e-736e90555d46\") " pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:14.679627 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.679610 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 18:45:14.680597 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.680578 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 18:45:14.692074 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.692058 2566 kubelet.go:1648] "Image garbage collection succeeded" Sep 16 18:45:14.710892 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.710870 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l8lv\" (UniqueName: \"kubernetes.io/projected/2656427d-2f81-43ab-9f1e-736e90555d46-kube-api-access-2l8lv\") pod \"keda-operator-579c8958f7-vrcqf\" (UID: \"2656427d-2f81-43ab-9f1e-736e90555d46\") " pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:14.710974 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.710897 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2656427d-2f81-43ab-9f1e-736e90555d46-certificates\") pod \"keda-operator-579c8958f7-vrcqf\" (UID: \"2656427d-2f81-43ab-9f1e-736e90555d46\") " pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:14.710974 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.710922 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/2656427d-2f81-43ab-9f1e-736e90555d46-cabundle0\") pod \"keda-operator-579c8958f7-vrcqf\" (UID: \"2656427d-2f81-43ab-9f1e-736e90555d46\") " pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:14.713375 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.713357 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Sep 16 18:45:14.713464 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.713377 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Sep 16 18:45:14.721314 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:45:14.721196 2566 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found Sep 16 18:45:14.721314 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:45:14.721224 2566 secret.go:281] references non-existent secret key: ca.crt Sep 16 18:45:14.721314 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:45:14.721235 2566 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Sep 16 18:45:14.721314 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:45:14.721251 2566 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-vrcqf: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Sep 16 18:45:14.734364 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:45:14.721353 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2656427d-2f81-43ab-9f1e-736e90555d46-certificates podName:2656427d-2f81-43ab-9f1e-736e90555d46 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:15.221329706 +0000 UTC m=+300.902184337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/2656427d-2f81-43ab-9f1e-736e90555d46-certificates") pod "keda-operator-579c8958f7-vrcqf" (UID: "2656427d-2f81-43ab-9f1e-736e90555d46") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Sep 16 18:45:14.734364 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.721625 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l8lv\" (UniqueName: \"kubernetes.io/projected/2656427d-2f81-43ab-9f1e-736e90555d46-kube-api-access-2l8lv\") pod \"keda-operator-579c8958f7-vrcqf\" (UID: \"2656427d-2f81-43ab-9f1e-736e90555d46\") " pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:14.734364 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:14.721725 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/2656427d-2f81-43ab-9f1e-736e90555d46-cabundle0\") pod \"keda-operator-579c8958f7-vrcqf\" (UID: \"2656427d-2f81-43ab-9f1e-736e90555d46\") " pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:15.313775 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.313744 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2656427d-2f81-43ab-9f1e-736e90555d46-certificates\") pod \"keda-operator-579c8958f7-vrcqf\" (UID: \"2656427d-2f81-43ab-9f1e-736e90555d46\") " pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:15.314122 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:45:15.313841 2566 secret.go:281] references non-existent secret key: ca.crt Sep 16 18:45:15.314122 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:45:15.313855 2566 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Sep 16 18:45:15.314122 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:45:15.313863 2566 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-579c8958f7-vrcqf: references non-existent secret key: ca.crt Sep 16 18:45:15.314122 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:45:15.313915 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2656427d-2f81-43ab-9f1e-736e90555d46-certificates podName:2656427d-2f81-43ab-9f1e-736e90555d46 nodeName:}" failed. No retries permitted until 2026-09-16 18:45:16.313901812 +0000 UTC m=+301.994756435 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/2656427d-2f81-43ab-9f1e-736e90555d46-certificates") pod "keda-operator-579c8958f7-vrcqf" (UID: "2656427d-2f81-43ab-9f1e-736e90555d46") : references non-existent secret key: ca.crt Sep 16 18:45:15.512787 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.512758 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s"] Sep 16 18:45:15.515717 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.515699 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:15.517916 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.517896 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Sep 16 18:45:15.529023 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.529000 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s"] Sep 16 18:45:15.616285 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.616230 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f9f26fa7-322d-4579-aa59-af3f9f0beaf0-certificates\") pod \"keda-metrics-apiserver-77857b7b98-t5n8s\" (UID: \"f9f26fa7-322d-4579-aa59-af3f9f0beaf0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:15.616285 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.616267 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/f9f26fa7-322d-4579-aa59-af3f9f0beaf0-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-t5n8s\" (UID: \"f9f26fa7-322d-4579-aa59-af3f9f0beaf0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:15.616401 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.616335 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzgrf\" (UniqueName: \"kubernetes.io/projected/f9f26fa7-322d-4579-aa59-af3f9f0beaf0-kube-api-access-xzgrf\") pod \"keda-metrics-apiserver-77857b7b98-t5n8s\" (UID: \"f9f26fa7-322d-4579-aa59-af3f9f0beaf0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:15.703527 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.703502 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-xc7dm"] Sep 16 18:45:15.706589 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.706572 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" Sep 16 18:45:15.709601 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.709584 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Sep 16 18:45:15.716687 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.716667 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzgrf\" (UniqueName: \"kubernetes.io/projected/f9f26fa7-322d-4579-aa59-af3f9f0beaf0-kube-api-access-xzgrf\") pod \"keda-metrics-apiserver-77857b7b98-t5n8s\" (UID: \"f9f26fa7-322d-4579-aa59-af3f9f0beaf0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:15.716780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.716711 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f9f26fa7-322d-4579-aa59-af3f9f0beaf0-certificates\") pod \"keda-metrics-apiserver-77857b7b98-t5n8s\" (UID: \"f9f26fa7-322d-4579-aa59-af3f9f0beaf0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:15.716780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.716738 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/f9f26fa7-322d-4579-aa59-af3f9f0beaf0-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-t5n8s\" (UID: \"f9f26fa7-322d-4579-aa59-af3f9f0beaf0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:15.716780 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.716771 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcwth\" (UniqueName: \"kubernetes.io/projected/c1e4e381-18db-410c-a673-4580fa62b08b-kube-api-access-jcwth\") pod \"keda-admission-6c4d864b86-xc7dm\" (UID: \"c1e4e381-18db-410c-a673-4580fa62b08b\") " pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" Sep 16 18:45:15.716921 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.716801 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c1e4e381-18db-410c-a673-4580fa62b08b-certificates\") pod \"keda-admission-6c4d864b86-xc7dm\" (UID: \"c1e4e381-18db-410c-a673-4580fa62b08b\") " pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" Sep 16 18:45:15.717143 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.717125 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/f9f26fa7-322d-4579-aa59-af3f9f0beaf0-temp-vol\") pod \"keda-metrics-apiserver-77857b7b98-t5n8s\" (UID: \"f9f26fa7-322d-4579-aa59-af3f9f0beaf0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:15.719021 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.719005 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/f9f26fa7-322d-4579-aa59-af3f9f0beaf0-certificates\") pod \"keda-metrics-apiserver-77857b7b98-t5n8s\" (UID: \"f9f26fa7-322d-4579-aa59-af3f9f0beaf0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:15.740136 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.740113 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzgrf\" (UniqueName: \"kubernetes.io/projected/f9f26fa7-322d-4579-aa59-af3f9f0beaf0-kube-api-access-xzgrf\") pod \"keda-metrics-apiserver-77857b7b98-t5n8s\" (UID: \"f9f26fa7-322d-4579-aa59-af3f9f0beaf0\") " pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:15.741819 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.741796 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-xc7dm"] Sep 16 18:45:15.822742 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.822709 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcwth\" (UniqueName: \"kubernetes.io/projected/c1e4e381-18db-410c-a673-4580fa62b08b-kube-api-access-jcwth\") pod \"keda-admission-6c4d864b86-xc7dm\" (UID: \"c1e4e381-18db-410c-a673-4580fa62b08b\") " pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" Sep 16 18:45:15.822903 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.822771 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c1e4e381-18db-410c-a673-4580fa62b08b-certificates\") pod \"keda-admission-6c4d864b86-xc7dm\" (UID: \"c1e4e381-18db-410c-a673-4580fa62b08b\") " pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" Sep 16 18:45:15.825410 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.825387 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c1e4e381-18db-410c-a673-4580fa62b08b-certificates\") pod \"keda-admission-6c4d864b86-xc7dm\" (UID: \"c1e4e381-18db-410c-a673-4580fa62b08b\") " pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" Sep 16 18:45:15.827340 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.827319 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-nkdmt\"" Sep 16 18:45:15.835430 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.835411 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:15.839326 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:15.839302 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcwth\" (UniqueName: \"kubernetes.io/projected/c1e4e381-18db-410c-a673-4580fa62b08b-kube-api-access-jcwth\") pod \"keda-admission-6c4d864b86-xc7dm\" (UID: \"c1e4e381-18db-410c-a673-4580fa62b08b\") " pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" Sep 16 18:45:16.016395 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:16.016368 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" Sep 16 18:45:16.112353 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:16.112329 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s"] Sep 16 18:45:16.114787 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:45:16.114761 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f26fa7_322d_4579_aa59_af3f9f0beaf0.slice/crio-d694bb576f3c07f664219b32cf8efed4629eea0996a9d424af89e6d11f657cf0 WatchSource:0}: Error finding container d694bb576f3c07f664219b32cf8efed4629eea0996a9d424af89e6d11f657cf0: Status 404 returned error can't find the container with id d694bb576f3c07f664219b32cf8efed4629eea0996a9d424af89e6d11f657cf0 Sep 16 18:45:16.116123 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:16.116109 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 16 18:45:16.306368 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:16.306345 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-6c4d864b86-xc7dm"] Sep 16 18:45:16.308076 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:45:16.308052 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e4e381_18db_410c_a673_4580fa62b08b.slice/crio-1276c0535dcc925e8c1529b532e10e97871068375bcb9465cc94c7c0272ee9d3 WatchSource:0}: Error finding container 1276c0535dcc925e8c1529b532e10e97871068375bcb9465cc94c7c0272ee9d3: Status 404 returned error can't find the container with id 1276c0535dcc925e8c1529b532e10e97871068375bcb9465cc94c7c0272ee9d3 Sep 16 18:45:16.318373 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:16.318346 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" event={"ID":"c1e4e381-18db-410c-a673-4580fa62b08b","Type":"ContainerStarted","Data":"1276c0535dcc925e8c1529b532e10e97871068375bcb9465cc94c7c0272ee9d3"} Sep 16 18:45:16.319456 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:16.319436 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" event={"ID":"f9f26fa7-322d-4579-aa59-af3f9f0beaf0","Type":"ContainerStarted","Data":"d694bb576f3c07f664219b32cf8efed4629eea0996a9d424af89e6d11f657cf0"} Sep 16 18:45:16.326837 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:16.326817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2656427d-2f81-43ab-9f1e-736e90555d46-certificates\") pod \"keda-operator-579c8958f7-vrcqf\" (UID: \"2656427d-2f81-43ab-9f1e-736e90555d46\") " pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:16.328985 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:16.328960 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/2656427d-2f81-43ab-9f1e-736e90555d46-certificates\") pod \"keda-operator-579c8958f7-vrcqf\" (UID: \"2656427d-2f81-43ab-9f1e-736e90555d46\") " pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:16.338755 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:16.338737 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:16.614922 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:16.614898 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-579c8958f7-vrcqf"] Sep 16 18:45:16.616744 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:45:16.616716 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2656427d_2f81_43ab_9f1e_736e90555d46.slice/crio-f11712caeeedcdb79592e7ab973cae5ec2ade24a11aa70db5d7d92850fa57296 WatchSource:0}: Error finding container f11712caeeedcdb79592e7ab973cae5ec2ade24a11aa70db5d7d92850fa57296: Status 404 returned error can't find the container with id f11712caeeedcdb79592e7ab973cae5ec2ade24a11aa70db5d7d92850fa57296 Sep 16 18:45:17.325343 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:17.325301 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-vrcqf" event={"ID":"2656427d-2f81-43ab-9f1e-736e90555d46","Type":"ContainerStarted","Data":"f11712caeeedcdb79592e7ab973cae5ec2ade24a11aa70db5d7d92850fa57296"} Sep 16 18:45:18.332161 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:18.332128 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" event={"ID":"c1e4e381-18db-410c-a673-4580fa62b08b","Type":"ContainerStarted","Data":"d0348afd78007d602f5ffa3ac34a8fb85c5f4ae51a61e3ba195d87e7b92ac7b9"} Sep 16 18:45:18.332903 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:18.332881 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" Sep 16 18:45:18.354616 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:18.354573 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" podStartSLOduration=1.423403668 podStartE2EDuration="3.354557247s" totalImagesPullingTime="1.931153579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:18.352952494 +0000 UTC m=+304.033807126" watchObservedRunningTime="2026-09-16 18:45:18.354557247 +0000 UTC m=+304.035411894" Sep 16 18:45:20.345555 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:20.345520 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" event={"ID":"f9f26fa7-322d-4579-aa59-af3f9f0beaf0","Type":"ContainerStarted","Data":"db58fe4c7a6bf7e15a0f04bf8be04ddb5169456a6724b61bb430404186f33648"} Sep 16 18:45:20.346036 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:20.345678 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:20.368954 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:20.368899 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" podStartSLOduration=2.080918731 podStartE2EDuration="5.368877271s" totalImagesPullingTime="3.28795854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:20.368297797 +0000 UTC m=+306.049152429" watchObservedRunningTime="2026-09-16 18:45:20.368877271 +0000 UTC m=+306.049731903" Sep 16 18:45:21.351142 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:21.351108 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-579c8958f7-vrcqf" event={"ID":"2656427d-2f81-43ab-9f1e-736e90555d46","Type":"ContainerStarted","Data":"cfff1d1790a053b45213d50961330e7ec16d21970b4929befa6549459fd9d459"} Sep 16 18:45:21.351548 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:21.351353 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:45:21.372017 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:21.371973 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-579c8958f7-vrcqf" podStartSLOduration=3.385227323 podStartE2EDuration="7.371954811s" totalImagesPullingTime="3.986727488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:45:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:45:21.370727592 +0000 UTC m=+307.051582224" watchObservedRunningTime="2026-09-16 18:45:21.371954811 +0000 UTC m=+307.052809470" Sep 16 18:45:31.355158 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:31.355127 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-77857b7b98-t5n8s" Sep 16 18:45:35.315921 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:35.315894 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-566c9cd69c-r4s2q" Sep 16 18:45:40.348026 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:40.347992 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-6c4d864b86-xc7dm" Sep 16 18:45:42.357858 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:45:42.357830 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-579c8958f7-vrcqf" Sep 16 18:46:40.726151 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.726063 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-599b79d756-85j6l"] Sep 16 18:46:40.729612 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.729593 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n"] Sep 16 18:46:40.729732 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.729714 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:46:40.732070 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.732046 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Sep 16 18:46:40.732417 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.732398 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-255gq\"" Sep 16 18:46:40.732662 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.732642 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Sep 16 18:46:40.732777 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.732744 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" Sep 16 18:46:40.732974 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.732944 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Sep 16 18:46:40.734482 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.734464 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Sep 16 18:46:40.734768 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.734751 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-trv57\"" Sep 16 18:46:40.742531 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.742511 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n"] Sep 16 18:46:40.744601 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.744581 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-599b79d756-85j6l"] Sep 16 18:46:40.758635 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.758615 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-gkkxl"] Sep 16 18:46:40.761640 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.761622 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-gkkxl" Sep 16 18:46:40.763759 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.763742 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-znrv2\"" Sep 16 18:46:40.763832 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.763742 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Sep 16 18:46:40.770590 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.770569 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-gkkxl"] Sep 16 18:46:40.844522 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.844497 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h6qc\" (UniqueName: \"kubernetes.io/projected/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-kube-api-access-9h6qc\") pod \"kserve-controller-manager-599b79d756-85j6l\" (UID: \"b1ba8607-8529-4b45-bcb2-a0fede5cf2df\") " pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:46:40.844614 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.844528 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d303272-5a5d-4fb2-b0a8-a8a4199276ff-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-vbz5n\" (UID: \"1d303272-5a5d-4fb2-b0a8-a8a4199276ff\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" Sep 16 18:46:40.844614 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.844561 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjpf4\" (UniqueName: \"kubernetes.io/projected/1d303272-5a5d-4fb2-b0a8-a8a4199276ff-kube-api-access-fjpf4\") pod \"llmisvc-controller-manager-645f9c8fd6-vbz5n\" (UID: \"1d303272-5a5d-4fb2-b0a8-a8a4199276ff\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" Sep 16 18:46:40.844614 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.844605 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-cert\") pod \"kserve-controller-manager-599b79d756-85j6l\" (UID: \"b1ba8607-8529-4b45-bcb2-a0fede5cf2df\") " pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:46:40.945388 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.945362 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v8bg\" (UniqueName: \"kubernetes.io/projected/b22eef33-2325-449e-94e3-8801dca5db3d-kube-api-access-2v8bg\") pod \"seaweedfs-64568bcd49-gkkxl\" (UID: \"b22eef33-2325-449e-94e3-8801dca5db3d\") " pod="kserve/seaweedfs-64568bcd49-gkkxl" Sep 16 18:46:40.945479 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.945395 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h6qc\" (UniqueName: \"kubernetes.io/projected/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-kube-api-access-9h6qc\") pod \"kserve-controller-manager-599b79d756-85j6l\" (UID: \"b1ba8607-8529-4b45-bcb2-a0fede5cf2df\") " pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:46:40.945479 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.945412 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d303272-5a5d-4fb2-b0a8-a8a4199276ff-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-vbz5n\" (UID: \"1d303272-5a5d-4fb2-b0a8-a8a4199276ff\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" Sep 16 18:46:40.945479 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.945436 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjpf4\" (UniqueName: \"kubernetes.io/projected/1d303272-5a5d-4fb2-b0a8-a8a4199276ff-kube-api-access-fjpf4\") pod \"llmisvc-controller-manager-645f9c8fd6-vbz5n\" (UID: \"1d303272-5a5d-4fb2-b0a8-a8a4199276ff\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" Sep 16 18:46:40.945479 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.945452 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-cert\") pod \"kserve-controller-manager-599b79d756-85j6l\" (UID: \"b1ba8607-8529-4b45-bcb2-a0fede5cf2df\") " pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:46:40.945479 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.945471 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b22eef33-2325-449e-94e3-8801dca5db3d-data\") pod \"seaweedfs-64568bcd49-gkkxl\" (UID: \"b22eef33-2325-449e-94e3-8801dca5db3d\") " pod="kserve/seaweedfs-64568bcd49-gkkxl" Sep 16 18:46:40.945657 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:46:40.945573 2566 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Sep 16 18:46:40.945657 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:46:40.945627 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-cert podName:b1ba8607-8529-4b45-bcb2-a0fede5cf2df nodeName:}" failed. No retries permitted until 2026-09-16 18:46:41.445612642 +0000 UTC m=+387.126467275 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-cert") pod "kserve-controller-manager-599b79d756-85j6l" (UID: "b1ba8607-8529-4b45-bcb2-a0fede5cf2df") : secret "kserve-webhook-server-cert" not found Sep 16 18:46:40.947798 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.947772 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d303272-5a5d-4fb2-b0a8-a8a4199276ff-cert\") pod \"llmisvc-controller-manager-645f9c8fd6-vbz5n\" (UID: \"1d303272-5a5d-4fb2-b0a8-a8a4199276ff\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" Sep 16 18:46:40.956443 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.956422 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h6qc\" (UniqueName: \"kubernetes.io/projected/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-kube-api-access-9h6qc\") pod \"kserve-controller-manager-599b79d756-85j6l\" (UID: \"b1ba8607-8529-4b45-bcb2-a0fede5cf2df\") " pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:46:40.956614 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:40.956594 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjpf4\" (UniqueName: \"kubernetes.io/projected/1d303272-5a5d-4fb2-b0a8-a8a4199276ff-kube-api-access-fjpf4\") pod \"llmisvc-controller-manager-645f9c8fd6-vbz5n\" (UID: \"1d303272-5a5d-4fb2-b0a8-a8a4199276ff\") " pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" Sep 16 18:46:41.046015 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.045993 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v8bg\" (UniqueName: \"kubernetes.io/projected/b22eef33-2325-449e-94e3-8801dca5db3d-kube-api-access-2v8bg\") pod \"seaweedfs-64568bcd49-gkkxl\" (UID: \"b22eef33-2325-449e-94e3-8801dca5db3d\") " pod="kserve/seaweedfs-64568bcd49-gkkxl" Sep 16 18:46:41.046096 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.046040 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b22eef33-2325-449e-94e3-8801dca5db3d-data\") pod \"seaweedfs-64568bcd49-gkkxl\" (UID: \"b22eef33-2325-449e-94e3-8801dca5db3d\") " pod="kserve/seaweedfs-64568bcd49-gkkxl" Sep 16 18:46:41.046339 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.046324 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b22eef33-2325-449e-94e3-8801dca5db3d-data\") pod \"seaweedfs-64568bcd49-gkkxl\" (UID: \"b22eef33-2325-449e-94e3-8801dca5db3d\") " pod="kserve/seaweedfs-64568bcd49-gkkxl" Sep 16 18:46:41.049044 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.049022 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" Sep 16 18:46:41.056425 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.056408 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v8bg\" (UniqueName: \"kubernetes.io/projected/b22eef33-2325-449e-94e3-8801dca5db3d-kube-api-access-2v8bg\") pod \"seaweedfs-64568bcd49-gkkxl\" (UID: \"b22eef33-2325-449e-94e3-8801dca5db3d\") " pod="kserve/seaweedfs-64568bcd49-gkkxl" Sep 16 18:46:41.070169 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.070149 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-gkkxl" Sep 16 18:46:41.326654 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.326620 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n"] Sep 16 18:46:41.328721 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:46:41.328695 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1d303272_5a5d_4fb2_b0a8_a8a4199276ff.slice/crio-4deca71d1d1706ce803683d0eef70f3d93349e3d01a19a30f0ea2087a82cc468 WatchSource:0}: Error finding container 4deca71d1d1706ce803683d0eef70f3d93349e3d01a19a30f0ea2087a82cc468: Status 404 returned error can't find the container with id 4deca71d1d1706ce803683d0eef70f3d93349e3d01a19a30f0ea2087a82cc468 Sep 16 18:46:41.355591 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.355560 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-gkkxl"] Sep 16 18:46:41.356781 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:46:41.356761 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb22eef33_2325_449e_94e3_8801dca5db3d.slice/crio-8226015dd89ecbe5b923cf28b091b8b18e6013f840ed76ba69075d44df15eec6 WatchSource:0}: Error finding container 8226015dd89ecbe5b923cf28b091b8b18e6013f840ed76ba69075d44df15eec6: Status 404 returned error can't find the container with id 8226015dd89ecbe5b923cf28b091b8b18e6013f840ed76ba69075d44df15eec6 Sep 16 18:46:41.449273 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.449247 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-cert\") pod \"kserve-controller-manager-599b79d756-85j6l\" (UID: \"b1ba8607-8529-4b45-bcb2-a0fede5cf2df\") " pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:46:41.451531 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.451511 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-cert\") pod \"kserve-controller-manager-599b79d756-85j6l\" (UID: \"b1ba8607-8529-4b45-bcb2-a0fede5cf2df\") " pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:46:41.642397 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.642341 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:46:41.717231 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.717175 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-gkkxl" event={"ID":"b22eef33-2325-449e-94e3-8801dca5db3d","Type":"ContainerStarted","Data":"8226015dd89ecbe5b923cf28b091b8b18e6013f840ed76ba69075d44df15eec6"} Sep 16 18:46:41.718446 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.718423 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" event={"ID":"1d303272-5a5d-4fb2-b0a8-a8a4199276ff","Type":"ContainerStarted","Data":"4deca71d1d1706ce803683d0eef70f3d93349e3d01a19a30f0ea2087a82cc468"} Sep 16 18:46:41.916587 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:41.916450 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-599b79d756-85j6l"] Sep 16 18:46:41.920450 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:46:41.920419 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ba8607_8529_4b45_bcb2_a0fede5cf2df.slice/crio-bc82adfed0c06dd8b43f25905186cdc12b486e59eea778ea377ae78c8eea7ab1 WatchSource:0}: Error finding container bc82adfed0c06dd8b43f25905186cdc12b486e59eea778ea377ae78c8eea7ab1: Status 404 returned error can't find the container with id bc82adfed0c06dd8b43f25905186cdc12b486e59eea778ea377ae78c8eea7ab1 Sep 16 18:46:42.728460 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:42.728375 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-599b79d756-85j6l" event={"ID":"b1ba8607-8529-4b45-bcb2-a0fede5cf2df","Type":"ContainerStarted","Data":"bc82adfed0c06dd8b43f25905186cdc12b486e59eea778ea377ae78c8eea7ab1"} Sep 16 18:46:44.528112 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.528077 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dc8ff7968-ccrk4"] Sep 16 18:46:44.531940 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.531912 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.538825 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.538795 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dc8ff7968-ccrk4"] Sep 16 18:46:44.677156 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.677121 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66c6f54c-4f86-42a7-87fe-e2e8702579ea-console-serving-cert\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.677333 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.677217 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66c6f54c-4f86-42a7-87fe-e2e8702579ea-oauth-serving-cert\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.677333 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.677247 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66c6f54c-4f86-42a7-87fe-e2e8702579ea-console-config\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.677333 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.677270 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c6f54c-4f86-42a7-87fe-e2e8702579ea-trusted-ca-bundle\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.677450 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.677352 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66c6f54c-4f86-42a7-87fe-e2e8702579ea-service-ca\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.677450 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.677407 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66c6f54c-4f86-42a7-87fe-e2e8702579ea-console-oauth-config\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.677450 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.677432 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8r47\" (UniqueName: \"kubernetes.io/projected/66c6f54c-4f86-42a7-87fe-e2e8702579ea-kube-api-access-g8r47\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.778245 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.778143 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66c6f54c-4f86-42a7-87fe-e2e8702579ea-service-ca\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.778401 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.778245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66c6f54c-4f86-42a7-87fe-e2e8702579ea-console-oauth-config\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.778401 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.778282 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8r47\" (UniqueName: \"kubernetes.io/projected/66c6f54c-4f86-42a7-87fe-e2e8702579ea-kube-api-access-g8r47\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.778401 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.778335 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66c6f54c-4f86-42a7-87fe-e2e8702579ea-console-serving-cert\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.778401 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.778364 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66c6f54c-4f86-42a7-87fe-e2e8702579ea-oauth-serving-cert\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.778401 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.778392 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66c6f54c-4f86-42a7-87fe-e2e8702579ea-console-config\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.778659 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.778414 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c6f54c-4f86-42a7-87fe-e2e8702579ea-trusted-ca-bundle\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.779044 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.779014 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/66c6f54c-4f86-42a7-87fe-e2e8702579ea-service-ca\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.779301 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.779270 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/66c6f54c-4f86-42a7-87fe-e2e8702579ea-oauth-serving-cert\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.779397 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.779301 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/66c6f54c-4f86-42a7-87fe-e2e8702579ea-console-config\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.779544 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.779512 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66c6f54c-4f86-42a7-87fe-e2e8702579ea-trusted-ca-bundle\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.781174 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.781154 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/66c6f54c-4f86-42a7-87fe-e2e8702579ea-console-oauth-config\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.781455 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.781430 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/66c6f54c-4f86-42a7-87fe-e2e8702579ea-console-serving-cert\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.788360 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.788340 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8r47\" (UniqueName: \"kubernetes.io/projected/66c6f54c-4f86-42a7-87fe-e2e8702579ea-kube-api-access-g8r47\") pod \"console-6dc8ff7968-ccrk4\" (UID: \"66c6f54c-4f86-42a7-87fe-e2e8702579ea\") " pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:44.844635 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:44.844607 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:46.048170 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.047887 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dc8ff7968-ccrk4"] Sep 16 18:46:46.051051 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:46:46.051022 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c6f54c_4f86_42a7_87fe_e2e8702579ea.slice/crio-13967ab44ded7c6b3d02db650467d6ea269618ea696c19656d84451426c1f1a0 WatchSource:0}: Error finding container 13967ab44ded7c6b3d02db650467d6ea269618ea696c19656d84451426c1f1a0: Status 404 returned error can't find the container with id 13967ab44ded7c6b3d02db650467d6ea269618ea696c19656d84451426c1f1a0 Sep 16 18:46:46.750572 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.750539 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-599b79d756-85j6l" event={"ID":"b1ba8607-8529-4b45-bcb2-a0fede5cf2df","Type":"ContainerStarted","Data":"5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41"} Sep 16 18:46:46.750793 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.750615 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:46:46.752087 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.752061 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-gkkxl" event={"ID":"b22eef33-2325-449e-94e3-8801dca5db3d","Type":"ContainerStarted","Data":"673aa41082eb12e725c674d254081e3d35b4d24ef8bad9e3781d80734ef41c25"} Sep 16 18:46:46.752237 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.752117 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-gkkxl" Sep 16 18:46:46.753516 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.753491 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" event={"ID":"1d303272-5a5d-4fb2-b0a8-a8a4199276ff","Type":"ContainerStarted","Data":"374ab774335f6bca6794ef39b7d54e75764a6f1f409402be48f90217e782bd10"} Sep 16 18:46:46.753604 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.753533 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" Sep 16 18:46:46.755072 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.755052 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc8ff7968-ccrk4" event={"ID":"66c6f54c-4f86-42a7-87fe-e2e8702579ea","Type":"ContainerStarted","Data":"b737f65e39d83b20887671c2dca6df3bdf0353090f025b61ce348847b6aa7cf8"} Sep 16 18:46:46.755147 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.755085 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dc8ff7968-ccrk4" event={"ID":"66c6f54c-4f86-42a7-87fe-e2e8702579ea","Type":"ContainerStarted","Data":"13967ab44ded7c6b3d02db650467d6ea269618ea696c19656d84451426c1f1a0"} Sep 16 18:46:46.769963 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.769908 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-599b79d756-85j6l" podStartSLOduration=2.952994714 podStartE2EDuration="6.769889977s" totalImagesPullingTime="3.816895263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:46.768949964 +0000 UTC m=+392.449804589" watchObservedRunningTime="2026-09-16 18:46:46.769889977 +0000 UTC m=+392.450744614" Sep 16 18:46:46.786376 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.786340 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-gkkxl" podStartSLOduration=2.265703729 podStartE2EDuration="6.786327144s" totalImagesPullingTime="4.520623415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:46.785360269 +0000 UTC m=+392.466214904" watchObservedRunningTime="2026-09-16 18:46:46.786327144 +0000 UTC m=+392.467181772" Sep 16 18:46:46.805832 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.805794 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" podStartSLOduration=2.507097781 podStartE2EDuration="6.805780627s" totalImagesPullingTime="4.298682846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:46.804240728 +0000 UTC m=+392.485095359" watchObservedRunningTime="2026-09-16 18:46:46.805780627 +0000 UTC m=+392.486635259" Sep 16 18:46:46.823166 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:46.823130 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dc8ff7968-ccrk4" podStartSLOduration=2.823114247 podStartE2EDuration="2.823114247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:46:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:46:46.821516612 +0000 UTC m=+392.502371244" watchObservedRunningTime="2026-09-16 18:46:46.823114247 +0000 UTC m=+392.503968880" Sep 16 18:46:52.761647 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:52.761619 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-gkkxl" Sep 16 18:46:54.844843 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:54.844805 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:54.845248 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:54.844873 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:54.849754 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:54.849732 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:55.803834 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:55.803800 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dc8ff7968-ccrk4" Sep 16 18:46:55.866726 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:46:55.866695 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c58f5f75-bsw97"] Sep 16 18:47:17.761618 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:17.761575 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-645f9c8fd6-vbz5n" Sep 16 18:47:17.764725 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:17.764702 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:47:20.888559 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:20.888496 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c58f5f75-bsw97" podUID="201b3e0e-9b4d-45f7-b821-ad55e5239d7b" containerName="console" containerID="cri-o://32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3" gracePeriod=15 Sep 16 18:47:21.142979 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.142928 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:47:21.232790 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.232766 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdjkw\" (UniqueName: \"kubernetes.io/projected/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-kube-api-access-sdjkw\") pod \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " Sep 16 18:47:21.232898 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.232823 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-trusted-ca-bundle\") pod \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " Sep 16 18:47:21.232898 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.232855 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-config\") pod \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " Sep 16 18:47:21.232898 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.232877 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-oauth-serving-cert\") pod \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " Sep 16 18:47:21.233047 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.232900 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-service-ca\") pod \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " Sep 16 18:47:21.233047 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.232930 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-serving-cert\") pod \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " Sep 16 18:47:21.233047 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.232963 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-oauth-config\") pod \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\" (UID: \"201b3e0e-9b4d-45f7-b821-ad55e5239d7b\") " Sep 16 18:47:21.233371 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.233326 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "201b3e0e-9b4d-45f7-b821-ad55e5239d7b" (UID: "201b3e0e-9b4d-45f7-b821-ad55e5239d7b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 18:47:21.233371 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.233334 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-config" (OuterVolumeSpecName: "console-config") pod "201b3e0e-9b4d-45f7-b821-ad55e5239d7b" (UID: "201b3e0e-9b4d-45f7-b821-ad55e5239d7b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 18:47:21.233371 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.233359 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "201b3e0e-9b4d-45f7-b821-ad55e5239d7b" (UID: "201b3e0e-9b4d-45f7-b821-ad55e5239d7b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 18:47:21.233497 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.233374 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-service-ca" (OuterVolumeSpecName: "service-ca") pod "201b3e0e-9b4d-45f7-b821-ad55e5239d7b" (UID: "201b3e0e-9b4d-45f7-b821-ad55e5239d7b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 18:47:21.234944 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.234915 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-kube-api-access-sdjkw" (OuterVolumeSpecName: "kube-api-access-sdjkw") pod "201b3e0e-9b4d-45f7-b821-ad55e5239d7b" (UID: "201b3e0e-9b4d-45f7-b821-ad55e5239d7b"). InnerVolumeSpecName "kube-api-access-sdjkw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 18:47:21.235060 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.234958 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "201b3e0e-9b4d-45f7-b821-ad55e5239d7b" (UID: "201b3e0e-9b4d-45f7-b821-ad55e5239d7b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 18:47:21.235060 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.234982 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "201b3e0e-9b4d-45f7-b821-ad55e5239d7b" (UID: "201b3e0e-9b4d-45f7-b821-ad55e5239d7b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 18:47:21.333464 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.333442 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-trusted-ca-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:47:21.333464 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.333465 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-config\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:47:21.333580 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.333474 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-oauth-serving-cert\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:47:21.333580 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.333484 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-service-ca\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:47:21.333580 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.333492 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-serving-cert\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:47:21.333580 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.333500 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-console-oauth-config\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:47:21.333580 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.333508 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdjkw\" (UniqueName: \"kubernetes.io/projected/201b3e0e-9b4d-45f7-b821-ad55e5239d7b-kube-api-access-sdjkw\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:47:21.914717 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.914679 2566 generic.go:356] "Generic (PLEG): container finished" podID="201b3e0e-9b4d-45f7-b821-ad55e5239d7b" containerID="32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3" exitCode=0 Sep 16 18:47:21.915113 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.914728 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c58f5f75-bsw97" Sep 16 18:47:21.915113 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.914764 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c58f5f75-bsw97" event={"ID":"201b3e0e-9b4d-45f7-b821-ad55e5239d7b","Type":"ContainerDied","Data":"32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3"} Sep 16 18:47:21.915113 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.914818 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c58f5f75-bsw97" event={"ID":"201b3e0e-9b4d-45f7-b821-ad55e5239d7b","Type":"ContainerDied","Data":"9963f0aa9ab907dc0064df1cb34d11b8b97a4bd812cc7a18ebf38717e58cd3d3"} Sep 16 18:47:21.915113 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.914841 2566 scope.go:117] "RemoveContainer" containerID="32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3" Sep 16 18:47:21.925035 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.925015 2566 scope.go:117] "RemoveContainer" containerID="32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3" Sep 16 18:47:21.925382 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:47:21.925362 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3\": container with ID starting with 32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3 not found: ID does not exist" containerID="32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3" Sep 16 18:47:21.925458 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.925391 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3"} err="failed to get container status \"32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3\": rpc error: code = NotFound desc = could not find container \"32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3\": container with ID starting with 32af4aaa20e5849b34a0a8bd1ea4e8d478956f71c5174796b8454be247230cc3 not found: ID does not exist" Sep 16 18:47:21.944155 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.944126 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c58f5f75-bsw97"] Sep 16 18:47:21.945651 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:21.945628 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c58f5f75-bsw97"] Sep 16 18:47:22.802762 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:22.802726 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="201b3e0e-9b4d-45f7-b821-ad55e5239d7b" path="/var/lib/kubelet/pods/201b3e0e-9b4d-45f7-b821-ad55e5239d7b/volumes" Sep 16 18:47:28.234535 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.234501 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-599b79d756-85j6l"] Sep 16 18:47:28.234899 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.234718 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-599b79d756-85j6l" podUID="b1ba8607-8529-4b45-bcb2-a0fede5cf2df" containerName="manager" containerID="cri-o://5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41" gracePeriod=10 Sep 16 18:47:28.294006 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.293979 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-599b79d756-jjvgv"] Sep 16 18:47:28.294378 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.294362 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="201b3e0e-9b4d-45f7-b821-ad55e5239d7b" containerName="console" Sep 16 18:47:28.294378 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.294379 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="201b3e0e-9b4d-45f7-b821-ad55e5239d7b" containerName="console" Sep 16 18:47:28.294517 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.294465 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="201b3e0e-9b4d-45f7-b821-ad55e5239d7b" containerName="console" Sep 16 18:47:28.296333 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.296314 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-599b79d756-jjvgv" Sep 16 18:47:28.312699 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.312678 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-599b79d756-jjvgv"] Sep 16 18:47:28.381352 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.381321 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d25dd1a1-f373-4156-9f35-8867aa912667-cert\") pod \"kserve-controller-manager-599b79d756-jjvgv\" (UID: \"d25dd1a1-f373-4156-9f35-8867aa912667\") " pod="kserve/kserve-controller-manager-599b79d756-jjvgv" Sep 16 18:47:28.381492 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.381417 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v9gz\" (UniqueName: \"kubernetes.io/projected/d25dd1a1-f373-4156-9f35-8867aa912667-kube-api-access-6v9gz\") pod \"kserve-controller-manager-599b79d756-jjvgv\" (UID: \"d25dd1a1-f373-4156-9f35-8867aa912667\") " pod="kserve/kserve-controller-manager-599b79d756-jjvgv" Sep 16 18:47:28.472324 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.472301 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:47:28.482291 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.482270 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v9gz\" (UniqueName: \"kubernetes.io/projected/d25dd1a1-f373-4156-9f35-8867aa912667-kube-api-access-6v9gz\") pod \"kserve-controller-manager-599b79d756-jjvgv\" (UID: \"d25dd1a1-f373-4156-9f35-8867aa912667\") " pod="kserve/kserve-controller-manager-599b79d756-jjvgv" Sep 16 18:47:28.482406 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.482324 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d25dd1a1-f373-4156-9f35-8867aa912667-cert\") pod \"kserve-controller-manager-599b79d756-jjvgv\" (UID: \"d25dd1a1-f373-4156-9f35-8867aa912667\") " pod="kserve/kserve-controller-manager-599b79d756-jjvgv" Sep 16 18:47:28.484341 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.484323 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d25dd1a1-f373-4156-9f35-8867aa912667-cert\") pod \"kserve-controller-manager-599b79d756-jjvgv\" (UID: \"d25dd1a1-f373-4156-9f35-8867aa912667\") " pod="kserve/kserve-controller-manager-599b79d756-jjvgv" Sep 16 18:47:28.495960 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.495907 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v9gz\" (UniqueName: \"kubernetes.io/projected/d25dd1a1-f373-4156-9f35-8867aa912667-kube-api-access-6v9gz\") pod \"kserve-controller-manager-599b79d756-jjvgv\" (UID: \"d25dd1a1-f373-4156-9f35-8867aa912667\") " pod="kserve/kserve-controller-manager-599b79d756-jjvgv" Sep 16 18:47:28.582812 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.582783 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9h6qc\" (UniqueName: \"kubernetes.io/projected/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-kube-api-access-9h6qc\") pod \"b1ba8607-8529-4b45-bcb2-a0fede5cf2df\" (UID: \"b1ba8607-8529-4b45-bcb2-a0fede5cf2df\") " Sep 16 18:47:28.582946 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.582818 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-cert\") pod \"b1ba8607-8529-4b45-bcb2-a0fede5cf2df\" (UID: \"b1ba8607-8529-4b45-bcb2-a0fede5cf2df\") " Sep 16 18:47:28.584879 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.584845 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-cert" (OuterVolumeSpecName: "cert") pod "b1ba8607-8529-4b45-bcb2-a0fede5cf2df" (UID: "b1ba8607-8529-4b45-bcb2-a0fede5cf2df"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 18:47:28.584963 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.584883 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-kube-api-access-9h6qc" (OuterVolumeSpecName: "kube-api-access-9h6qc") pod "b1ba8607-8529-4b45-bcb2-a0fede5cf2df" (UID: "b1ba8607-8529-4b45-bcb2-a0fede5cf2df"). InnerVolumeSpecName "kube-api-access-9h6qc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 16 18:47:28.618233 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.618212 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-599b79d756-jjvgv" Sep 16 18:47:28.683702 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.683676 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9h6qc\" (UniqueName: \"kubernetes.io/projected/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-kube-api-access-9h6qc\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:47:28.683814 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.683705 2566 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b1ba8607-8529-4b45-bcb2-a0fede5cf2df-cert\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:47:28.905599 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.905571 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-599b79d756-jjvgv"] Sep 16 18:47:28.908488 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:47:28.908464 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd25dd1a1_f373_4156_9f35_8867aa912667.slice/crio-ccaa03d85b0aef305fce6b52374e93e70ff9982cab001d93d64504ae0b142441 WatchSource:0}: Error finding container ccaa03d85b0aef305fce6b52374e93e70ff9982cab001d93d64504ae0b142441: Status 404 returned error can't find the container with id ccaa03d85b0aef305fce6b52374e93e70ff9982cab001d93d64504ae0b142441 Sep 16 18:47:28.949313 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.949284 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-599b79d756-jjvgv" event={"ID":"d25dd1a1-f373-4156-9f35-8867aa912667","Type":"ContainerStarted","Data":"ccaa03d85b0aef305fce6b52374e93e70ff9982cab001d93d64504ae0b142441"} Sep 16 18:47:28.950471 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.950450 2566 generic.go:356] "Generic (PLEG): container finished" podID="b1ba8607-8529-4b45-bcb2-a0fede5cf2df" containerID="5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41" exitCode=0 Sep 16 18:47:28.950564 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.950483 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-599b79d756-85j6l" event={"ID":"b1ba8607-8529-4b45-bcb2-a0fede5cf2df","Type":"ContainerDied","Data":"5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41"} Sep 16 18:47:28.950564 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.950491 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-599b79d756-85j6l" Sep 16 18:47:28.950564 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.950510 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-599b79d756-85j6l" event={"ID":"b1ba8607-8529-4b45-bcb2-a0fede5cf2df","Type":"ContainerDied","Data":"bc82adfed0c06dd8b43f25905186cdc12b486e59eea778ea377ae78c8eea7ab1"} Sep 16 18:47:28.950564 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.950530 2566 scope.go:117] "RemoveContainer" containerID="5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41" Sep 16 18:47:28.958239 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.958200 2566 scope.go:117] "RemoveContainer" containerID="5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41" Sep 16 18:47:28.958511 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:47:28.958492 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41\": container with ID starting with 5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41 not found: ID does not exist" containerID="5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41" Sep 16 18:47:28.958589 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.958518 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41"} err="failed to get container status \"5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41\": rpc error: code = NotFound desc = could not find container \"5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41\": container with ID starting with 5da2d09fe09ecdb55cd30867e4aed38062dd97ae2dacf85f3a5bc9b9f36bda41 not found: ID does not exist" Sep 16 18:47:28.980218 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.980196 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-599b79d756-85j6l"] Sep 16 18:47:28.986794 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:28.986772 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-599b79d756-85j6l"] Sep 16 18:47:29.957449 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:29.957418 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-599b79d756-jjvgv" event={"ID":"d25dd1a1-f373-4156-9f35-8867aa912667","Type":"ContainerStarted","Data":"6ffc4fcb335a0bb21fe5596ad37a893b6a12e920586f7007524e505c784037d3"} Sep 16 18:47:29.957798 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:29.957556 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-599b79d756-jjvgv" Sep 16 18:47:29.987200 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:29.987146 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-599b79d756-jjvgv" podStartSLOduration=1.6696174940000001 podStartE2EDuration="1.987131922s" totalImagesPullingTime="317.514428ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:47:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:47:29.986754843 +0000 UTC m=+435.667609475" watchObservedRunningTime="2026-09-16 18:47:29.987131922 +0000 UTC m=+435.667986556" Sep 16 18:47:30.797021 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:47:30.796982 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1ba8607-8529-4b45-bcb2-a0fede5cf2df" path="/var/lib/kubelet/pods/b1ba8607-8529-4b45-bcb2-a0fede5cf2df/volumes" Sep 16 18:48:00.967243 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:48:00.967205 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-599b79d756-jjvgv" Sep 16 18:48:14.707173 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:48:14.707143 2566 scope.go:117] "RemoveContainer" containerID="b492c222624c13a6ba1c87bd7b8e7804e8ce34f15c05663e661af6f2d10998d7" Sep 16 18:50:14.710264 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:50:14.710236 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 18:50:14.710769 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:50:14.710485 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 18:51:09.312964 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.312929 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt"] Sep 16 18:51:09.313448 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.313270 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1ba8607-8529-4b45-bcb2-a0fede5cf2df" containerName="manager" Sep 16 18:51:09.313448 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.313283 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1ba8607-8529-4b45-bcb2-a0fede5cf2df" containerName="manager" Sep 16 18:51:09.313448 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.313338 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1ba8607-8529-4b45-bcb2-a0fede5cf2df" containerName="manager" Sep 16 18:51:09.316236 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.316183 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:09.320343 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.320323 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-24wmq\"" Sep 16 18:51:09.321047 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.321026 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-35a06-serving-cert\"" Sep 16 18:51:09.321131 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.321045 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 16 18:51:09.321606 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.321588 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-35a06-kube-rbac-proxy-sar-config\"" Sep 16 18:51:09.328787 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.328767 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt"] Sep 16 18:51:09.385845 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.385818 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8849771-7bad-4bd9-a5ec-c34b805f3f8f-proxy-tls\") pod \"switch-graph-35a06-7fd9cccc6d-k2gnt\" (UID: \"b8849771-7bad-4bd9-a5ec-c34b805f3f8f\") " pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:09.385975 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.385958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8849771-7bad-4bd9-a5ec-c34b805f3f8f-openshift-service-ca-bundle\") pod \"switch-graph-35a06-7fd9cccc6d-k2gnt\" (UID: \"b8849771-7bad-4bd9-a5ec-c34b805f3f8f\") " pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:09.486353 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.486321 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8849771-7bad-4bd9-a5ec-c34b805f3f8f-openshift-service-ca-bundle\") pod \"switch-graph-35a06-7fd9cccc6d-k2gnt\" (UID: \"b8849771-7bad-4bd9-a5ec-c34b805f3f8f\") " pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:09.486496 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.486365 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8849771-7bad-4bd9-a5ec-c34b805f3f8f-proxy-tls\") pod \"switch-graph-35a06-7fd9cccc6d-k2gnt\" (UID: \"b8849771-7bad-4bd9-a5ec-c34b805f3f8f\") " pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:09.486937 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.486914 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8849771-7bad-4bd9-a5ec-c34b805f3f8f-openshift-service-ca-bundle\") pod \"switch-graph-35a06-7fd9cccc6d-k2gnt\" (UID: \"b8849771-7bad-4bd9-a5ec-c34b805f3f8f\") " pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:09.488620 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.488599 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8849771-7bad-4bd9-a5ec-c34b805f3f8f-proxy-tls\") pod \"switch-graph-35a06-7fd9cccc6d-k2gnt\" (UID: \"b8849771-7bad-4bd9-a5ec-c34b805f3f8f\") " pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:09.626334 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.626271 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:09.899725 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.899701 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt"] Sep 16 18:51:09.901915 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:51:09.901892 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8849771_7bad_4bd9_a5ec_c34b805f3f8f.slice/crio-0f90c341ef5c2a92463d039f975ad7c1457c36f20afa25b8b7f6bf12a8e96a82 WatchSource:0}: Error finding container 0f90c341ef5c2a92463d039f975ad7c1457c36f20afa25b8b7f6bf12a8e96a82: Status 404 returned error can't find the container with id 0f90c341ef5c2a92463d039f975ad7c1457c36f20afa25b8b7f6bf12a8e96a82 Sep 16 18:51:09.903775 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.903759 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 16 18:51:09.948655 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:09.948633 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" event={"ID":"b8849771-7bad-4bd9-a5ec-c34b805f3f8f","Type":"ContainerStarted","Data":"0f90c341ef5c2a92463d039f975ad7c1457c36f20afa25b8b7f6bf12a8e96a82"} Sep 16 18:51:11.958173 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:11.958141 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" event={"ID":"b8849771-7bad-4bd9-a5ec-c34b805f3f8f","Type":"ContainerStarted","Data":"8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30"} Sep 16 18:51:11.958519 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:11.958271 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:11.976122 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:11.976082 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" podStartSLOduration=1.037942782 podStartE2EDuration="2.976068147s" totalImagesPullingTime="1.938125365s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:51:11.975177396 +0000 UTC m=+657.656032042" watchObservedRunningTime="2026-09-16 18:51:11.976068147 +0000 UTC m=+657.656922779" Sep 16 18:51:17.967726 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:17.967695 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:19.521489 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:19.521452 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt"] Sep 16 18:51:19.521849 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:19.521718 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" podUID="b8849771-7bad-4bd9-a5ec-c34b805f3f8f" containerName="switch-graph-35a06" containerID="cri-o://8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30" gracePeriod=30 Sep 16 18:51:22.966555 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:22.966507 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" podUID="b8849771-7bad-4bd9-a5ec-c34b805f3f8f" containerName="switch-graph-35a06" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:51:27.967268 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:27.967217 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" podUID="b8849771-7bad-4bd9-a5ec-c34b805f3f8f" containerName="switch-graph-35a06" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:51:32.966568 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:32.966523 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" podUID="b8849771-7bad-4bd9-a5ec-c34b805f3f8f" containerName="switch-graph-35a06" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:51:32.967005 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:32.966653 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:37.966164 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:37.966127 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" podUID="b8849771-7bad-4bd9-a5ec-c34b805f3f8f" containerName="switch-graph-35a06" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:51:42.966270 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:42.966227 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" podUID="b8849771-7bad-4bd9-a5ec-c34b805f3f8f" containerName="switch-graph-35a06" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:51:47.966527 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:47.966485 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" podUID="b8849771-7bad-4bd9-a5ec-c34b805f3f8f" containerName="switch-graph-35a06" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:51:49.677121 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:49.677097 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:49.789759 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:49.789690 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8849771-7bad-4bd9-a5ec-c34b805f3f8f-openshift-service-ca-bundle\") pod \"b8849771-7bad-4bd9-a5ec-c34b805f3f8f\" (UID: \"b8849771-7bad-4bd9-a5ec-c34b805f3f8f\") " Sep 16 18:51:49.789759 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:49.789756 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8849771-7bad-4bd9-a5ec-c34b805f3f8f-proxy-tls\") pod \"b8849771-7bad-4bd9-a5ec-c34b805f3f8f\" (UID: \"b8849771-7bad-4bd9-a5ec-c34b805f3f8f\") " Sep 16 18:51:49.790040 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:49.790013 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8849771-7bad-4bd9-a5ec-c34b805f3f8f-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b8849771-7bad-4bd9-a5ec-c34b805f3f8f" (UID: "b8849771-7bad-4bd9-a5ec-c34b805f3f8f"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 18:51:49.791865 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:49.791843 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8849771-7bad-4bd9-a5ec-c34b805f3f8f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b8849771-7bad-4bd9-a5ec-c34b805f3f8f" (UID: "b8849771-7bad-4bd9-a5ec-c34b805f3f8f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 18:51:49.890908 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:49.890878 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8849771-7bad-4bd9-a5ec-c34b805f3f8f-openshift-service-ca-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:51:49.890908 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:49.890906 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b8849771-7bad-4bd9-a5ec-c34b805f3f8f-proxy-tls\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:51:50.126056 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:50.125984 2566 generic.go:356] "Generic (PLEG): container finished" podID="b8849771-7bad-4bd9-a5ec-c34b805f3f8f" containerID="8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30" exitCode=0 Sep 16 18:51:50.126056 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:50.126027 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" Sep 16 18:51:50.126206 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:50.126074 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" event={"ID":"b8849771-7bad-4bd9-a5ec-c34b805f3f8f","Type":"ContainerDied","Data":"8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30"} Sep 16 18:51:50.126206 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:50.126125 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt" event={"ID":"b8849771-7bad-4bd9-a5ec-c34b805f3f8f","Type":"ContainerDied","Data":"0f90c341ef5c2a92463d039f975ad7c1457c36f20afa25b8b7f6bf12a8e96a82"} Sep 16 18:51:50.126206 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:50.126151 2566 scope.go:117] "RemoveContainer" containerID="8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30" Sep 16 18:51:50.135225 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:50.135183 2566 scope.go:117] "RemoveContainer" containerID="8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30" Sep 16 18:51:50.135501 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:51:50.135476 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30\": container with ID starting with 8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30 not found: ID does not exist" containerID="8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30" Sep 16 18:51:50.135578 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:50.135515 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30"} err="failed to get container status \"8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30\": rpc error: code = NotFound desc = could not find container \"8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30\": container with ID starting with 8acb5439229799f96ed7455c6b443c13fc5de308a62206bffd43aa027c531e30 not found: ID does not exist" Sep 16 18:51:50.147729 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:50.147709 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt"] Sep 16 18:51:50.150066 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:50.150043 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-35a06-7fd9cccc6d-k2gnt"] Sep 16 18:51:50.796818 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:50.796787 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8849771-7bad-4bd9-a5ec-c34b805f3f8f" path="/var/lib/kubelet/pods/b8849771-7bad-4bd9-a5ec-c34b805f3f8f/volumes" Sep 16 18:51:59.262047 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.262012 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs"] Sep 16 18:51:59.262434 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.262374 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8849771-7bad-4bd9-a5ec-c34b805f3f8f" containerName="switch-graph-35a06" Sep 16 18:51:59.262434 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.262387 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8849771-7bad-4bd9-a5ec-c34b805f3f8f" containerName="switch-graph-35a06" Sep 16 18:51:59.262508 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.262445 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8849771-7bad-4bd9-a5ec-c34b805f3f8f" containerName="switch-graph-35a06" Sep 16 18:51:59.266615 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.266594 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:51:59.268711 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.268687 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Sep 16 18:51:59.268711 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.268697 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\"" Sep 16 18:51:59.268860 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.268692 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 16 18:51:59.268969 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.268956 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-24wmq\"" Sep 16 18:51:59.274943 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.274919 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs"] Sep 16 18:51:59.358686 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.358651 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6861e96a-7f2e-4976-9832-8eed217fdbac-proxy-tls\") pod \"model-chainer-7c9bd794bb-kw2xs\" (UID: \"6861e96a-7f2e-4976-9832-8eed217fdbac\") " pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:51:59.358821 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.358705 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6861e96a-7f2e-4976-9832-8eed217fdbac-openshift-service-ca-bundle\") pod \"model-chainer-7c9bd794bb-kw2xs\" (UID: \"6861e96a-7f2e-4976-9832-8eed217fdbac\") " pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:51:59.459106 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.459077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6861e96a-7f2e-4976-9832-8eed217fdbac-proxy-tls\") pod \"model-chainer-7c9bd794bb-kw2xs\" (UID: \"6861e96a-7f2e-4976-9832-8eed217fdbac\") " pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:51:59.459255 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.459127 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6861e96a-7f2e-4976-9832-8eed217fdbac-openshift-service-ca-bundle\") pod \"model-chainer-7c9bd794bb-kw2xs\" (UID: \"6861e96a-7f2e-4976-9832-8eed217fdbac\") " pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:51:59.459725 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.459696 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6861e96a-7f2e-4976-9832-8eed217fdbac-openshift-service-ca-bundle\") pod \"model-chainer-7c9bd794bb-kw2xs\" (UID: \"6861e96a-7f2e-4976-9832-8eed217fdbac\") " pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:51:59.461395 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.461363 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6861e96a-7f2e-4976-9832-8eed217fdbac-proxy-tls\") pod \"model-chainer-7c9bd794bb-kw2xs\" (UID: \"6861e96a-7f2e-4976-9832-8eed217fdbac\") " pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:51:59.577145 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.577082 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:51:59.866170 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:51:59.866050 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs"] Sep 16 18:51:59.871202 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:51:59.871155 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6861e96a_7f2e_4976_9832_8eed217fdbac.slice/crio-f1ee13d0f2d7d1d634321c6e5e54158906e6ae46fbc59c7dc2a4e1282f3e32e2 WatchSource:0}: Error finding container f1ee13d0f2d7d1d634321c6e5e54158906e6ae46fbc59c7dc2a4e1282f3e32e2: Status 404 returned error can't find the container with id f1ee13d0f2d7d1d634321c6e5e54158906e6ae46fbc59c7dc2a4e1282f3e32e2 Sep 16 18:52:00.175594 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:00.175513 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" event={"ID":"6861e96a-7f2e-4976-9832-8eed217fdbac","Type":"ContainerStarted","Data":"8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd"} Sep 16 18:52:00.175594 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:00.175571 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" event={"ID":"6861e96a-7f2e-4976-9832-8eed217fdbac","Type":"ContainerStarted","Data":"f1ee13d0f2d7d1d634321c6e5e54158906e6ae46fbc59c7dc2a4e1282f3e32e2"} Sep 16 18:52:00.175807 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:00.175659 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:52:00.195905 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:00.195860 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" podStartSLOduration=1.1958461360000001 podStartE2EDuration="1.195846136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:51:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:52:00.194399599 +0000 UTC m=+705.875254231" watchObservedRunningTime="2026-09-16 18:52:00.195846136 +0000 UTC m=+705.876700768" Sep 16 18:52:06.186031 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:06.186003 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:52:09.358276 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:09.358175 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs"] Sep 16 18:52:09.359032 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:09.358416 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" podUID="6861e96a-7f2e-4976-9832-8eed217fdbac" containerName="model-chainer" containerID="cri-o://8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd" gracePeriod=30 Sep 16 18:52:11.185039 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:11.185001 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" podUID="6861e96a-7f2e-4976-9832-8eed217fdbac" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:52:16.184814 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:16.184775 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" podUID="6861e96a-7f2e-4976-9832-8eed217fdbac" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:52:19.767271 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:19.767239 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6"] Sep 16 18:52:19.771730 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:19.771709 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 18:52:19.773956 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:19.773933 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-15f71-kube-rbac-proxy-sar-config\"" Sep 16 18:52:19.774064 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:19.773935 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-15f71-serving-cert\"" Sep 16 18:52:19.778815 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:19.778796 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6"] Sep 16 18:52:19.814926 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:19.814899 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/afbfba8a-e006-4e8b-8f46-f07e36266776-proxy-tls\") pod \"switch-graph-15f71-6d86bc76cd-7x6g6\" (UID: \"afbfba8a-e006-4e8b-8f46-f07e36266776\") " pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 18:52:19.815018 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:19.814953 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afbfba8a-e006-4e8b-8f46-f07e36266776-openshift-service-ca-bundle\") pod \"switch-graph-15f71-6d86bc76cd-7x6g6\" (UID: \"afbfba8a-e006-4e8b-8f46-f07e36266776\") " pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 18:52:19.915514 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:19.915484 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/afbfba8a-e006-4e8b-8f46-f07e36266776-proxy-tls\") pod \"switch-graph-15f71-6d86bc76cd-7x6g6\" (UID: \"afbfba8a-e006-4e8b-8f46-f07e36266776\") " pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 18:52:19.915685 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:19.915571 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afbfba8a-e006-4e8b-8f46-f07e36266776-openshift-service-ca-bundle\") pod \"switch-graph-15f71-6d86bc76cd-7x6g6\" (UID: \"afbfba8a-e006-4e8b-8f46-f07e36266776\") " pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 18:52:19.916256 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:19.916228 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afbfba8a-e006-4e8b-8f46-f07e36266776-openshift-service-ca-bundle\") pod \"switch-graph-15f71-6d86bc76cd-7x6g6\" (UID: \"afbfba8a-e006-4e8b-8f46-f07e36266776\") " pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 18:52:19.917772 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:19.917745 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/afbfba8a-e006-4e8b-8f46-f07e36266776-proxy-tls\") pod \"switch-graph-15f71-6d86bc76cd-7x6g6\" (UID: \"afbfba8a-e006-4e8b-8f46-f07e36266776\") " pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 18:52:20.082504 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:20.082423 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 18:52:20.355132 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:20.355059 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6"] Sep 16 18:52:20.358225 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:52:20.358180 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafbfba8a_e006_4e8b_8f46_f07e36266776.slice/crio-30aced4a1f551ef6a96a6ad5a680d90f7634a9e7b9bcf6351851550e008d711c WatchSource:0}: Error finding container 30aced4a1f551ef6a96a6ad5a680d90f7634a9e7b9bcf6351851550e008d711c: Status 404 returned error can't find the container with id 30aced4a1f551ef6a96a6ad5a680d90f7634a9e7b9bcf6351851550e008d711c Sep 16 18:52:21.184315 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:21.184271 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" podUID="6861e96a-7f2e-4976-9832-8eed217fdbac" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:52:21.184746 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:21.184379 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:52:21.269258 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:21.269225 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" event={"ID":"afbfba8a-e006-4e8b-8f46-f07e36266776","Type":"ContainerStarted","Data":"e91498f60811a061b86e164fd770eb6312f1019cb0bad6dc1310d0f785aa3a36"} Sep 16 18:52:21.269390 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:21.269272 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" event={"ID":"afbfba8a-e006-4e8b-8f46-f07e36266776","Type":"ContainerStarted","Data":"30aced4a1f551ef6a96a6ad5a680d90f7634a9e7b9bcf6351851550e008d711c"} Sep 16 18:52:21.269390 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:21.269330 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 18:52:21.287409 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:21.287367 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" podStartSLOduration=2.287355284 podStartE2EDuration="2.287355284s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:52:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:52:21.286530639 +0000 UTC m=+726.967385272" watchObservedRunningTime="2026-09-16 18:52:21.287355284 +0000 UTC m=+726.968209993" Sep 16 18:52:26.184635 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:26.184598 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" podUID="6861e96a-7f2e-4976-9832-8eed217fdbac" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:52:27.280167 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:27.280133 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 18:52:31.184456 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:31.184414 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" podUID="6861e96a-7f2e-4976-9832-8eed217fdbac" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:52:36.184881 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:36.184839 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" podUID="6861e96a-7f2e-4976-9832-8eed217fdbac" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 18:52:39.503028 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:39.503002 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:52:39.554675 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:39.554651 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6861e96a-7f2e-4976-9832-8eed217fdbac-openshift-service-ca-bundle\") pod \"6861e96a-7f2e-4976-9832-8eed217fdbac\" (UID: \"6861e96a-7f2e-4976-9832-8eed217fdbac\") " Sep 16 18:52:39.554778 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:39.554697 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6861e96a-7f2e-4976-9832-8eed217fdbac-proxy-tls\") pod \"6861e96a-7f2e-4976-9832-8eed217fdbac\" (UID: \"6861e96a-7f2e-4976-9832-8eed217fdbac\") " Sep 16 18:52:39.554983 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:39.554961 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6861e96a-7f2e-4976-9832-8eed217fdbac-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "6861e96a-7f2e-4976-9832-8eed217fdbac" (UID: "6861e96a-7f2e-4976-9832-8eed217fdbac"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 18:52:39.556614 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:39.556596 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6861e96a-7f2e-4976-9832-8eed217fdbac-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "6861e96a-7f2e-4976-9832-8eed217fdbac" (UID: "6861e96a-7f2e-4976-9832-8eed217fdbac"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 18:52:39.656091 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:39.656041 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/6861e96a-7f2e-4976-9832-8eed217fdbac-proxy-tls\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:52:39.656091 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:39.656064 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6861e96a-7f2e-4976-9832-8eed217fdbac-openshift-service-ca-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 18:52:40.358770 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:40.358731 2566 generic.go:356] "Generic (PLEG): container finished" podID="6861e96a-7f2e-4976-9832-8eed217fdbac" containerID="8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd" exitCode=0 Sep 16 18:52:40.359008 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:40.358784 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" event={"ID":"6861e96a-7f2e-4976-9832-8eed217fdbac","Type":"ContainerDied","Data":"8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd"} Sep 16 18:52:40.359008 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:40.358793 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" Sep 16 18:52:40.359008 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:40.358836 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs" event={"ID":"6861e96a-7f2e-4976-9832-8eed217fdbac","Type":"ContainerDied","Data":"f1ee13d0f2d7d1d634321c6e5e54158906e6ae46fbc59c7dc2a4e1282f3e32e2"} Sep 16 18:52:40.359008 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:40.358866 2566 scope.go:117] "RemoveContainer" containerID="8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd" Sep 16 18:52:40.367431 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:40.367253 2566 scope.go:117] "RemoveContainer" containerID="8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd" Sep 16 18:52:40.367623 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:52:40.367600 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd\": container with ID starting with 8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd not found: ID does not exist" containerID="8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd" Sep 16 18:52:40.367664 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:40.367635 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd"} err="failed to get container status \"8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd\": rpc error: code = NotFound desc = could not find container \"8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd\": container with ID starting with 8278864500ac240ca9205ca5a6ed851ef5c1390349c290696fc5a5820f4f78bd not found: ID does not exist" Sep 16 18:52:40.382539 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:40.382510 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs"] Sep 16 18:52:40.385101 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:40.385078 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-7c9bd794bb-kw2xs"] Sep 16 18:52:40.796807 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:52:40.796775 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6861e96a-7f2e-4976-9832-8eed217fdbac" path="/var/lib/kubelet/pods/6861e96a-7f2e-4976-9832-8eed217fdbac/volumes" Sep 16 18:53:09.584088 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.584053 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k"] Sep 16 18:53:09.585071 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.585022 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6861e96a-7f2e-4976-9832-8eed217fdbac" containerName="model-chainer" Sep 16 18:53:09.585071 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.585069 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6861e96a-7f2e-4976-9832-8eed217fdbac" containerName="model-chainer" Sep 16 18:53:09.585325 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.585177 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6861e96a-7f2e-4976-9832-8eed217fdbac" containerName="model-chainer" Sep 16 18:53:09.589569 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.589549 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 18:53:09.591643 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.591618 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-21301-kube-rbac-proxy-sar-config\"" Sep 16 18:53:09.591643 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.591620 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-21301-serving-cert\"" Sep 16 18:53:09.597230 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.597207 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k"] Sep 16 18:53:09.674416 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.674388 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/70d8ffbb-2d35-4378-8756-5d5d015fb708-proxy-tls\") pod \"sequence-graph-21301-7fb87c6d96-cn98k\" (UID: \"70d8ffbb-2d35-4378-8756-5d5d015fb708\") " pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 18:53:09.674538 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.674475 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70d8ffbb-2d35-4378-8756-5d5d015fb708-openshift-service-ca-bundle\") pod \"sequence-graph-21301-7fb87c6d96-cn98k\" (UID: \"70d8ffbb-2d35-4378-8756-5d5d015fb708\") " pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 18:53:09.775391 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.775359 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/70d8ffbb-2d35-4378-8756-5d5d015fb708-proxy-tls\") pod \"sequence-graph-21301-7fb87c6d96-cn98k\" (UID: \"70d8ffbb-2d35-4378-8756-5d5d015fb708\") " pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 18:53:09.775522 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.775410 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70d8ffbb-2d35-4378-8756-5d5d015fb708-openshift-service-ca-bundle\") pod \"sequence-graph-21301-7fb87c6d96-cn98k\" (UID: \"70d8ffbb-2d35-4378-8756-5d5d015fb708\") " pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 18:53:09.775522 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:53:09.775501 2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-21301-serving-cert: secret "sequence-graph-21301-serving-cert" not found Sep 16 18:53:09.775597 ip-10-0-133-99 kubenswrapper[2566]: E0916 18:53:09.775575 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70d8ffbb-2d35-4378-8756-5d5d015fb708-proxy-tls podName:70d8ffbb-2d35-4378-8756-5d5d015fb708 nodeName:}" failed. No retries permitted until 2026-09-16 18:53:10.27556065 +0000 UTC m=+775.956415260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/70d8ffbb-2d35-4378-8756-5d5d015fb708-proxy-tls") pod "sequence-graph-21301-7fb87c6d96-cn98k" (UID: "70d8ffbb-2d35-4378-8756-5d5d015fb708") : secret "sequence-graph-21301-serving-cert" not found Sep 16 18:53:09.775927 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:09.775910 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70d8ffbb-2d35-4378-8756-5d5d015fb708-openshift-service-ca-bundle\") pod \"sequence-graph-21301-7fb87c6d96-cn98k\" (UID: \"70d8ffbb-2d35-4378-8756-5d5d015fb708\") " pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 18:53:10.279609 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:10.279566 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/70d8ffbb-2d35-4378-8756-5d5d015fb708-proxy-tls\") pod \"sequence-graph-21301-7fb87c6d96-cn98k\" (UID: \"70d8ffbb-2d35-4378-8756-5d5d015fb708\") " pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 18:53:10.282347 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:10.282320 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/70d8ffbb-2d35-4378-8756-5d5d015fb708-proxy-tls\") pod \"sequence-graph-21301-7fb87c6d96-cn98k\" (UID: \"70d8ffbb-2d35-4378-8756-5d5d015fb708\") " pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 18:53:10.499731 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:10.499702 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 18:53:10.777654 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:10.777624 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k"] Sep 16 18:53:10.780540 ip-10-0-133-99 kubenswrapper[2566]: W0916 18:53:10.780509 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70d8ffbb_2d35_4378_8756_5d5d015fb708.slice/crio-a42aa213fa7767757170e20e16d56d35ffcfcb299919b474dd23d859a1c8ff25 WatchSource:0}: Error finding container a42aa213fa7767757170e20e16d56d35ffcfcb299919b474dd23d859a1c8ff25: Status 404 returned error can't find the container with id a42aa213fa7767757170e20e16d56d35ffcfcb299919b474dd23d859a1c8ff25 Sep 16 18:53:11.507135 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:11.507099 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" event={"ID":"70d8ffbb-2d35-4378-8756-5d5d015fb708","Type":"ContainerStarted","Data":"7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb"} Sep 16 18:53:11.507450 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:11.507160 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" event={"ID":"70d8ffbb-2d35-4378-8756-5d5d015fb708","Type":"ContainerStarted","Data":"a42aa213fa7767757170e20e16d56d35ffcfcb299919b474dd23d859a1c8ff25"} Sep 16 18:53:11.507450 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:11.507329 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 18:53:11.530136 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:11.530086 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" podStartSLOduration=2.530072704 podStartE2EDuration="2.530072704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 18:53:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 18:53:11.528304458 +0000 UTC m=+777.209159090" watchObservedRunningTime="2026-09-16 18:53:11.530072704 +0000 UTC m=+777.210927335" Sep 16 18:53:17.518095 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:53:17.518062 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 18:55:14.737723 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:55:14.737648 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 18:55:14.739271 ip-10-0-133-99 kubenswrapper[2566]: I0916 18:55:14.739245 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:00:14.764400 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:00:14.764371 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:00:14.767794 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:00:14.767772 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:00:34.535128 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:00:34.535095 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6"] Sep 16 19:00:34.535509 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:00:34.535426 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" podUID="afbfba8a-e006-4e8b-8f46-f07e36266776" containerName="switch-graph-15f71" containerID="cri-o://e91498f60811a061b86e164fd770eb6312f1019cb0bad6dc1310d0f785aa3a36" gracePeriod=30 Sep 16 19:00:37.278417 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:00:37.278378 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" podUID="afbfba8a-e006-4e8b-8f46-f07e36266776" containerName="switch-graph-15f71" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:00:42.278541 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:00:42.278497 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" podUID="afbfba8a-e006-4e8b-8f46-f07e36266776" containerName="switch-graph-15f71" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:00:47.278214 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:00:47.278148 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" podUID="afbfba8a-e006-4e8b-8f46-f07e36266776" containerName="switch-graph-15f71" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:00:47.278742 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:00:47.278309 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 19:00:52.278844 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:00:52.278792 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" podUID="afbfba8a-e006-4e8b-8f46-f07e36266776" containerName="switch-graph-15f71" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:00:57.278571 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:00:57.278516 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" podUID="afbfba8a-e006-4e8b-8f46-f07e36266776" containerName="switch-graph-15f71" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:01:02.278845 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:02.278801 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" podUID="afbfba8a-e006-4e8b-8f46-f07e36266776" containerName="switch-graph-15f71" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:01:04.673333 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:04.673302 2566 generic.go:356] "Generic (PLEG): container finished" podID="afbfba8a-e006-4e8b-8f46-f07e36266776" containerID="e91498f60811a061b86e164fd770eb6312f1019cb0bad6dc1310d0f785aa3a36" exitCode=0 Sep 16 19:01:04.673650 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:04.673355 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" event={"ID":"afbfba8a-e006-4e8b-8f46-f07e36266776","Type":"ContainerDied","Data":"e91498f60811a061b86e164fd770eb6312f1019cb0bad6dc1310d0f785aa3a36"} Sep 16 19:01:04.673650 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:04.673408 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" event={"ID":"afbfba8a-e006-4e8b-8f46-f07e36266776","Type":"ContainerDied","Data":"30aced4a1f551ef6a96a6ad5a680d90f7634a9e7b9bcf6351851550e008d711c"} Sep 16 19:01:04.673650 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:04.673433 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="30aced4a1f551ef6a96a6ad5a680d90f7634a9e7b9bcf6351851550e008d711c" Sep 16 19:01:04.681291 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:04.681273 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 19:01:04.761845 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:04.761818 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afbfba8a-e006-4e8b-8f46-f07e36266776-openshift-service-ca-bundle\") pod \"afbfba8a-e006-4e8b-8f46-f07e36266776\" (UID: \"afbfba8a-e006-4e8b-8f46-f07e36266776\") " Sep 16 19:01:04.761947 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:04.761861 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/afbfba8a-e006-4e8b-8f46-f07e36266776-proxy-tls\") pod \"afbfba8a-e006-4e8b-8f46-f07e36266776\" (UID: \"afbfba8a-e006-4e8b-8f46-f07e36266776\") " Sep 16 19:01:04.762171 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:04.762147 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afbfba8a-e006-4e8b-8f46-f07e36266776-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "afbfba8a-e006-4e8b-8f46-f07e36266776" (UID: "afbfba8a-e006-4e8b-8f46-f07e36266776"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 19:01:04.763831 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:04.763808 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afbfba8a-e006-4e8b-8f46-f07e36266776-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "afbfba8a-e006-4e8b-8f46-f07e36266776" (UID: "afbfba8a-e006-4e8b-8f46-f07e36266776"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 19:01:04.863204 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:04.863160 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/afbfba8a-e006-4e8b-8f46-f07e36266776-proxy-tls\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:01:04.863204 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:04.863183 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afbfba8a-e006-4e8b-8f46-f07e36266776-openshift-service-ca-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:01:05.678204 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:05.678165 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6" Sep 16 19:01:05.718927 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:05.718897 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6"] Sep 16 19:01:05.722502 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:05.722475 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-15f71-6d86bc76cd-7x6g6"] Sep 16 19:01:06.796068 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:06.796035 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afbfba8a-e006-4e8b-8f46-f07e36266776" path="/var/lib/kubelet/pods/afbfba8a-e006-4e8b-8f46-f07e36266776/volumes" Sep 16 19:01:14.819407 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:14.819377 2566 scope.go:117] "RemoveContainer" containerID="e91498f60811a061b86e164fd770eb6312f1019cb0bad6dc1310d0f785aa3a36" Sep 16 19:01:24.090875 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:24.090794 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k"] Sep 16 19:01:24.091449 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:24.091094 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" podUID="70d8ffbb-2d35-4378-8756-5d5d015fb708" containerName="sequence-graph-21301" containerID="cri-o://7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb" gracePeriod=30 Sep 16 19:01:27.516448 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:27.516399 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" podUID="70d8ffbb-2d35-4378-8756-5d5d015fb708" containerName="sequence-graph-21301" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:01:32.516563 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:32.516518 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" podUID="70d8ffbb-2d35-4378-8756-5d5d015fb708" containerName="sequence-graph-21301" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:01:34.816346 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.816311 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84"] Sep 16 19:01:34.816796 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.816773 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afbfba8a-e006-4e8b-8f46-f07e36266776" containerName="switch-graph-15f71" Sep 16 19:01:34.816872 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.816800 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="afbfba8a-e006-4e8b-8f46-f07e36266776" containerName="switch-graph-15f71" Sep 16 19:01:34.816930 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.816907 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="afbfba8a-e006-4e8b-8f46-f07e36266776" containerName="switch-graph-15f71" Sep 16 19:01:34.821364 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.821343 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:01:34.824373 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.824350 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-1593f-serving-cert\"" Sep 16 19:01:34.824451 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.824383 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-1593f-kube-rbac-proxy-sar-config\"" Sep 16 19:01:34.831247 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.831226 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84"] Sep 16 19:01:34.875800 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.875772 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/885974b1-c3cc-47fb-8946-c9c60a2b8fed-openshift-service-ca-bundle\") pod \"ensemble-graph-1593f-5c79fd86d5-9xg84\" (UID: \"885974b1-c3cc-47fb-8946-c9c60a2b8fed\") " pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:01:34.875905 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.875806 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/885974b1-c3cc-47fb-8946-c9c60a2b8fed-proxy-tls\") pod \"ensemble-graph-1593f-5c79fd86d5-9xg84\" (UID: \"885974b1-c3cc-47fb-8946-c9c60a2b8fed\") " pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:01:34.977072 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.977040 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/885974b1-c3cc-47fb-8946-c9c60a2b8fed-openshift-service-ca-bundle\") pod \"ensemble-graph-1593f-5c79fd86d5-9xg84\" (UID: \"885974b1-c3cc-47fb-8946-c9c60a2b8fed\") " pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:01:34.977236 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.977078 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/885974b1-c3cc-47fb-8946-c9c60a2b8fed-proxy-tls\") pod \"ensemble-graph-1593f-5c79fd86d5-9xg84\" (UID: \"885974b1-c3cc-47fb-8946-c9c60a2b8fed\") " pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:01:34.977654 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.977629 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/885974b1-c3cc-47fb-8946-c9c60a2b8fed-openshift-service-ca-bundle\") pod \"ensemble-graph-1593f-5c79fd86d5-9xg84\" (UID: \"885974b1-c3cc-47fb-8946-c9c60a2b8fed\") " pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:01:34.979396 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:34.979370 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/885974b1-c3cc-47fb-8946-c9c60a2b8fed-proxy-tls\") pod \"ensemble-graph-1593f-5c79fd86d5-9xg84\" (UID: \"885974b1-c3cc-47fb-8946-c9c60a2b8fed\") " pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:01:35.132248 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:35.132181 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:01:35.426782 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:35.426710 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84"] Sep 16 19:01:35.429718 ip-10-0-133-99 kubenswrapper[2566]: W0916 19:01:35.429681 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod885974b1_c3cc_47fb_8946_c9c60a2b8fed.slice/crio-e0e9880b4ca18efd976845b10b8ab07c0839afdd196c12accf1464600d72f27b WatchSource:0}: Error finding container e0e9880b4ca18efd976845b10b8ab07c0839afdd196c12accf1464600d72f27b: Status 404 returned error can't find the container with id e0e9880b4ca18efd976845b10b8ab07c0839afdd196c12accf1464600d72f27b Sep 16 19:01:35.815183 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:35.815149 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" event={"ID":"885974b1-c3cc-47fb-8946-c9c60a2b8fed","Type":"ContainerStarted","Data":"9734787fd65d263fce271b2a5a8dd6aa3997413c5df46e160f5b50976815625d"} Sep 16 19:01:35.815367 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:35.815216 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" event={"ID":"885974b1-c3cc-47fb-8946-c9c60a2b8fed","Type":"ContainerStarted","Data":"e0e9880b4ca18efd976845b10b8ab07c0839afdd196c12accf1464600d72f27b"} Sep 16 19:01:35.815367 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:35.815272 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:01:35.837929 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:35.837885 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" podStartSLOduration=1.837871915 podStartE2EDuration="1.837871915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:01:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:01:35.834815674 +0000 UTC m=+1281.515670305" watchObservedRunningTime="2026-09-16 19:01:35.837871915 +0000 UTC m=+1281.518726563" Sep 16 19:01:37.516251 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:37.516210 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" podUID="70d8ffbb-2d35-4378-8756-5d5d015fb708" containerName="sequence-graph-21301" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:01:37.516604 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:37.516308 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 19:01:41.825639 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:41.825606 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:01:42.516242 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:42.516205 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" podUID="70d8ffbb-2d35-4378-8756-5d5d015fb708" containerName="sequence-graph-21301" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:01:44.902726 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:44.902641 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84"] Sep 16 19:01:44.903156 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:44.902909 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" podUID="885974b1-c3cc-47fb-8946-c9c60a2b8fed" containerName="ensemble-graph-1593f" containerID="cri-o://9734787fd65d263fce271b2a5a8dd6aa3997413c5df46e160f5b50976815625d" gracePeriod=30 Sep 16 19:01:46.824128 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:46.824085 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" podUID="885974b1-c3cc-47fb-8946-c9c60a2b8fed" containerName="ensemble-graph-1593f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:01:47.516780 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:47.516742 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" podUID="70d8ffbb-2d35-4378-8756-5d5d015fb708" containerName="sequence-graph-21301" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:01:51.823625 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:51.823579 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" podUID="885974b1-c3cc-47fb-8946-c9c60a2b8fed" containerName="ensemble-graph-1593f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:01:52.516040 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:52.515992 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" podUID="70d8ffbb-2d35-4378-8756-5d5d015fb708" containerName="sequence-graph-21301" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:01:54.231264 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.231244 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 19:01:54.315450 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.315426 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70d8ffbb-2d35-4378-8756-5d5d015fb708-openshift-service-ca-bundle\") pod \"70d8ffbb-2d35-4378-8756-5d5d015fb708\" (UID: \"70d8ffbb-2d35-4378-8756-5d5d015fb708\") " Sep 16 19:01:54.315567 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.315519 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/70d8ffbb-2d35-4378-8756-5d5d015fb708-proxy-tls\") pod \"70d8ffbb-2d35-4378-8756-5d5d015fb708\" (UID: \"70d8ffbb-2d35-4378-8756-5d5d015fb708\") " Sep 16 19:01:54.315745 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.315724 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70d8ffbb-2d35-4378-8756-5d5d015fb708-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "70d8ffbb-2d35-4378-8756-5d5d015fb708" (UID: "70d8ffbb-2d35-4378-8756-5d5d015fb708"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 19:01:54.317454 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.317435 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70d8ffbb-2d35-4378-8756-5d5d015fb708-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "70d8ffbb-2d35-4378-8756-5d5d015fb708" (UID: "70d8ffbb-2d35-4378-8756-5d5d015fb708"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 19:01:54.416208 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.416146 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/70d8ffbb-2d35-4378-8756-5d5d015fb708-proxy-tls\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:01:54.416208 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.416169 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/70d8ffbb-2d35-4378-8756-5d5d015fb708-openshift-service-ca-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:01:54.898662 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.898623 2566 generic.go:356] "Generic (PLEG): container finished" podID="70d8ffbb-2d35-4378-8756-5d5d015fb708" containerID="7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb" exitCode=0 Sep 16 19:01:54.898886 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.898669 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" Sep 16 19:01:54.898886 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.898676 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" event={"ID":"70d8ffbb-2d35-4378-8756-5d5d015fb708","Type":"ContainerDied","Data":"7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb"} Sep 16 19:01:54.898886 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.898723 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k" event={"ID":"70d8ffbb-2d35-4378-8756-5d5d015fb708","Type":"ContainerDied","Data":"a42aa213fa7767757170e20e16d56d35ffcfcb299919b474dd23d859a1c8ff25"} Sep 16 19:01:54.898886 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.898747 2566 scope.go:117] "RemoveContainer" containerID="7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb" Sep 16 19:01:54.906597 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.906580 2566 scope.go:117] "RemoveContainer" containerID="7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb" Sep 16 19:01:54.906861 ip-10-0-133-99 kubenswrapper[2566]: E0916 19:01:54.906844 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb\": container with ID starting with 7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb not found: ID does not exist" containerID="7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb" Sep 16 19:01:54.906913 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.906892 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb"} err="failed to get container status \"7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb\": rpc error: code = NotFound desc = could not find container \"7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb\": container with ID starting with 7600a1fe46795e488c29c74249f0aec994e3006162d90ae7b80ff6cc4e5279fb not found: ID does not exist" Sep 16 19:01:54.938904 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.938884 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k"] Sep 16 19:01:54.951897 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:54.951875 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-21301-7fb87c6d96-cn98k"] Sep 16 19:01:56.797312 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:56.797272 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70d8ffbb-2d35-4378-8756-5d5d015fb708" path="/var/lib/kubelet/pods/70d8ffbb-2d35-4378-8756-5d5d015fb708/volumes" Sep 16 19:01:56.823400 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:56.823370 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" podUID="885974b1-c3cc-47fb-8946-c9c60a2b8fed" containerName="ensemble-graph-1593f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:01:56.823537 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:01:56.823457 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:02:01.824406 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:01.824360 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" podUID="885974b1-c3cc-47fb-8946-c9c60a2b8fed" containerName="ensemble-graph-1593f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:02:06.823451 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:06.823411 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" podUID="885974b1-c3cc-47fb-8946-c9c60a2b8fed" containerName="ensemble-graph-1593f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:02:11.824270 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:11.824228 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" podUID="885974b1-c3cc-47fb-8946-c9c60a2b8fed" containerName="ensemble-graph-1593f" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:02:14.987388 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:14.987357 2566 generic.go:356] "Generic (PLEG): container finished" podID="885974b1-c3cc-47fb-8946-c9c60a2b8fed" containerID="9734787fd65d263fce271b2a5a8dd6aa3997413c5df46e160f5b50976815625d" exitCode=137 Sep 16 19:02:14.987736 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:14.987402 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" event={"ID":"885974b1-c3cc-47fb-8946-c9c60a2b8fed","Type":"ContainerDied","Data":"9734787fd65d263fce271b2a5a8dd6aa3997413c5df46e160f5b50976815625d"} Sep 16 19:02:15.081027 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:15.080999 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:02:15.179076 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:15.179013 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/885974b1-c3cc-47fb-8946-c9c60a2b8fed-openshift-service-ca-bundle\") pod \"885974b1-c3cc-47fb-8946-c9c60a2b8fed\" (UID: \"885974b1-c3cc-47fb-8946-c9c60a2b8fed\") " Sep 16 19:02:15.179210 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:15.179133 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/885974b1-c3cc-47fb-8946-c9c60a2b8fed-proxy-tls\") pod \"885974b1-c3cc-47fb-8946-c9c60a2b8fed\" (UID: \"885974b1-c3cc-47fb-8946-c9c60a2b8fed\") " Sep 16 19:02:15.179420 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:15.179399 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/885974b1-c3cc-47fb-8946-c9c60a2b8fed-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "885974b1-c3cc-47fb-8946-c9c60a2b8fed" (UID: "885974b1-c3cc-47fb-8946-c9c60a2b8fed"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 19:02:15.180945 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:15.180929 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/885974b1-c3cc-47fb-8946-c9c60a2b8fed-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "885974b1-c3cc-47fb-8946-c9c60a2b8fed" (UID: "885974b1-c3cc-47fb-8946-c9c60a2b8fed"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 19:02:15.280179 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:15.280146 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/885974b1-c3cc-47fb-8946-c9c60a2b8fed-proxy-tls\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:02:15.280179 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:15.280175 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/885974b1-c3cc-47fb-8946-c9c60a2b8fed-openshift-service-ca-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:02:15.992543 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:15.992509 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" event={"ID":"885974b1-c3cc-47fb-8946-c9c60a2b8fed","Type":"ContainerDied","Data":"e0e9880b4ca18efd976845b10b8ab07c0839afdd196c12accf1464600d72f27b"} Sep 16 19:02:15.992972 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:15.992565 2566 scope.go:117] "RemoveContainer" containerID="9734787fd65d263fce271b2a5a8dd6aa3997413c5df46e160f5b50976815625d" Sep 16 19:02:15.992972 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:15.992524 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84" Sep 16 19:02:16.018314 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:16.018285 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84"] Sep 16 19:02:16.025607 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:16.025585 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-1593f-5c79fd86d5-9xg84"] Sep 16 19:02:16.802642 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:16.802594 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="885974b1-c3cc-47fb-8946-c9c60a2b8fed" path="/var/lib/kubelet/pods/885974b1-c3cc-47fb-8946-c9c60a2b8fed/volumes" Sep 16 19:02:24.367869 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.367836 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp"] Sep 16 19:02:24.368388 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.368181 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70d8ffbb-2d35-4378-8756-5d5d015fb708" containerName="sequence-graph-21301" Sep 16 19:02:24.368388 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.368208 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="70d8ffbb-2d35-4378-8756-5d5d015fb708" containerName="sequence-graph-21301" Sep 16 19:02:24.368388 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.368219 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="885974b1-c3cc-47fb-8946-c9c60a2b8fed" containerName="ensemble-graph-1593f" Sep 16 19:02:24.368388 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.368225 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="885974b1-c3cc-47fb-8946-c9c60a2b8fed" containerName="ensemble-graph-1593f" Sep 16 19:02:24.368388 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.368274 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="70d8ffbb-2d35-4378-8756-5d5d015fb708" containerName="sequence-graph-21301" Sep 16 19:02:24.368388 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.368283 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="885974b1-c3cc-47fb-8946-c9c60a2b8fed" containerName="ensemble-graph-1593f" Sep 16 19:02:24.372607 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.372589 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:02:24.375198 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.375163 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-34033-serving-cert\"" Sep 16 19:02:24.375320 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.375252 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-34033-kube-rbac-proxy-sar-config\"" Sep 16 19:02:24.375582 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.375564 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-24wmq\"" Sep 16 19:02:24.375693 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.375672 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 16 19:02:24.385504 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.385483 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp"] Sep 16 19:02:24.552103 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.552065 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b062018-80b8-42fb-83ca-57cfb9fd805e-openshift-service-ca-bundle\") pod \"sequence-graph-34033-5489c4744b-q5rmp\" (UID: \"1b062018-80b8-42fb-83ca-57cfb9fd805e\") " pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:02:24.552292 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.552133 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b062018-80b8-42fb-83ca-57cfb9fd805e-proxy-tls\") pod \"sequence-graph-34033-5489c4744b-q5rmp\" (UID: \"1b062018-80b8-42fb-83ca-57cfb9fd805e\") " pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:02:24.653491 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.653409 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b062018-80b8-42fb-83ca-57cfb9fd805e-proxy-tls\") pod \"sequence-graph-34033-5489c4744b-q5rmp\" (UID: \"1b062018-80b8-42fb-83ca-57cfb9fd805e\") " pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:02:24.653491 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.653483 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b062018-80b8-42fb-83ca-57cfb9fd805e-openshift-service-ca-bundle\") pod \"sequence-graph-34033-5489c4744b-q5rmp\" (UID: \"1b062018-80b8-42fb-83ca-57cfb9fd805e\") " pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:02:24.654060 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.654043 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b062018-80b8-42fb-83ca-57cfb9fd805e-openshift-service-ca-bundle\") pod \"sequence-graph-34033-5489c4744b-q5rmp\" (UID: \"1b062018-80b8-42fb-83ca-57cfb9fd805e\") " pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:02:24.655806 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.655786 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b062018-80b8-42fb-83ca-57cfb9fd805e-proxy-tls\") pod \"sequence-graph-34033-5489c4744b-q5rmp\" (UID: \"1b062018-80b8-42fb-83ca-57cfb9fd805e\") " pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:02:24.683909 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.683879 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:02:24.967952 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:24.967873 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp"] Sep 16 19:02:24.971955 ip-10-0-133-99 kubenswrapper[2566]: W0916 19:02:24.971928 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b062018_80b8_42fb_83ca_57cfb9fd805e.slice/crio-7fe6dc2500ee2055006fcd4a8db1429311752ab0c0f34b1f8cd21864b1a59f0a WatchSource:0}: Error finding container 7fe6dc2500ee2055006fcd4a8db1429311752ab0c0f34b1f8cd21864b1a59f0a: Status 404 returned error can't find the container with id 7fe6dc2500ee2055006fcd4a8db1429311752ab0c0f34b1f8cd21864b1a59f0a Sep 16 19:02:25.032589 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:25.032548 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" event={"ID":"1b062018-80b8-42fb-83ca-57cfb9fd805e","Type":"ContainerStarted","Data":"7fe6dc2500ee2055006fcd4a8db1429311752ab0c0f34b1f8cd21864b1a59f0a"} Sep 16 19:02:26.038345 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:26.038308 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" event={"ID":"1b062018-80b8-42fb-83ca-57cfb9fd805e","Type":"ContainerStarted","Data":"335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515"} Sep 16 19:02:26.038741 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:26.038433 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:02:26.061852 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:26.061807 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" podStartSLOduration=2.061793258 podStartE2EDuration="2.061793258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:02:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:02:26.060674784 +0000 UTC m=+1331.741529416" watchObservedRunningTime="2026-09-16 19:02:26.061793258 +0000 UTC m=+1331.742647931" Sep 16 19:02:32.048416 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:32.048380 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:02:34.494846 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:34.494802 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp"] Sep 16 19:02:34.495234 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:34.494975 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" podUID="1b062018-80b8-42fb-83ca-57cfb9fd805e" containerName="sequence-graph-34033" containerID="cri-o://335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515" gracePeriod=30 Sep 16 19:02:37.047548 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:37.047465 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" podUID="1b062018-80b8-42fb-83ca-57cfb9fd805e" containerName="sequence-graph-34033" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:02:42.046386 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:42.046352 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" podUID="1b062018-80b8-42fb-83ca-57cfb9fd805e" containerName="sequence-graph-34033" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:02:45.185618 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:45.185588 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz"] Sep 16 19:02:45.188655 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:45.188637 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:02:45.191281 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:45.191259 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-e4920-serving-cert\"" Sep 16 19:02:45.192115 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:45.192098 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-e4920-kube-rbac-proxy-sar-config\"" Sep 16 19:02:45.206488 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:45.206467 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz"] Sep 16 19:02:45.295705 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:45.295678 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0207330b-1a61-406e-be72-947ec2a29d0c-proxy-tls\") pod \"ensemble-graph-e4920-59fb4569b6-fl2qz\" (UID: \"0207330b-1a61-406e-be72-947ec2a29d0c\") " pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:02:45.295819 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:45.295720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0207330b-1a61-406e-be72-947ec2a29d0c-openshift-service-ca-bundle\") pod \"ensemble-graph-e4920-59fb4569b6-fl2qz\" (UID: \"0207330b-1a61-406e-be72-947ec2a29d0c\") " pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:02:45.396334 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:45.396292 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0207330b-1a61-406e-be72-947ec2a29d0c-openshift-service-ca-bundle\") pod \"ensemble-graph-e4920-59fb4569b6-fl2qz\" (UID: \"0207330b-1a61-406e-be72-947ec2a29d0c\") " pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:02:45.396484 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:45.396376 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0207330b-1a61-406e-be72-947ec2a29d0c-proxy-tls\") pod \"ensemble-graph-e4920-59fb4569b6-fl2qz\" (UID: \"0207330b-1a61-406e-be72-947ec2a29d0c\") " pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:02:45.396484 ip-10-0-133-99 kubenswrapper[2566]: E0916 19:02:45.396475 2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-e4920-serving-cert: secret "ensemble-graph-e4920-serving-cert" not found Sep 16 19:02:45.396564 ip-10-0-133-99 kubenswrapper[2566]: E0916 19:02:45.396536 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0207330b-1a61-406e-be72-947ec2a29d0c-proxy-tls podName:0207330b-1a61-406e-be72-947ec2a29d0c nodeName:}" failed. No retries permitted until 2026-09-16 19:02:45.89652026 +0000 UTC m=+1351.577374870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/0207330b-1a61-406e-be72-947ec2a29d0c-proxy-tls") pod "ensemble-graph-e4920-59fb4569b6-fl2qz" (UID: "0207330b-1a61-406e-be72-947ec2a29d0c") : secret "ensemble-graph-e4920-serving-cert" not found Sep 16 19:02:45.396920 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:45.396901 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0207330b-1a61-406e-be72-947ec2a29d0c-openshift-service-ca-bundle\") pod \"ensemble-graph-e4920-59fb4569b6-fl2qz\" (UID: \"0207330b-1a61-406e-be72-947ec2a29d0c\") " pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:02:45.900837 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:45.900806 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0207330b-1a61-406e-be72-947ec2a29d0c-proxy-tls\") pod \"ensemble-graph-e4920-59fb4569b6-fl2qz\" (UID: \"0207330b-1a61-406e-be72-947ec2a29d0c\") " pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:02:45.903321 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:45.903291 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0207330b-1a61-406e-be72-947ec2a29d0c-proxy-tls\") pod \"ensemble-graph-e4920-59fb4569b6-fl2qz\" (UID: \"0207330b-1a61-406e-be72-947ec2a29d0c\") " pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:02:46.098974 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:46.098949 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:02:46.384231 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:46.384209 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz"] Sep 16 19:02:46.386153 ip-10-0-133-99 kubenswrapper[2566]: W0916 19:02:46.386127 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0207330b_1a61_406e_be72_947ec2a29d0c.slice/crio-5d45e33aa937b7c1c7a5dcd787c7ec2747e36ca552e05e7557bd4622ab55f47d WatchSource:0}: Error finding container 5d45e33aa937b7c1c7a5dcd787c7ec2747e36ca552e05e7557bd4622ab55f47d: Status 404 returned error can't find the container with id 5d45e33aa937b7c1c7a5dcd787c7ec2747e36ca552e05e7557bd4622ab55f47d Sep 16 19:02:47.047058 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:47.047023 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" podUID="1b062018-80b8-42fb-83ca-57cfb9fd805e" containerName="sequence-graph-34033" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:02:47.047257 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:47.047125 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:02:47.135300 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:47.135269 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" event={"ID":"0207330b-1a61-406e-be72-947ec2a29d0c","Type":"ContainerStarted","Data":"b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33"} Sep 16 19:02:47.135434 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:47.135311 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" event={"ID":"0207330b-1a61-406e-be72-947ec2a29d0c","Type":"ContainerStarted","Data":"5d45e33aa937b7c1c7a5dcd787c7ec2747e36ca552e05e7557bd4622ab55f47d"} Sep 16 19:02:47.135434 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:47.135370 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:02:47.183488 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:47.183448 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" podStartSLOduration=2.18343492 podStartE2EDuration="2.18343492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:02:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:02:47.182789969 +0000 UTC m=+1352.863644597" watchObservedRunningTime="2026-09-16 19:02:47.18343492 +0000 UTC m=+1352.864289551" Sep 16 19:02:52.047451 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:52.047402 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" podUID="1b062018-80b8-42fb-83ca-57cfb9fd805e" containerName="sequence-graph-34033" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:02:53.145981 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:53.145954 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:02:57.047448 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:02:57.047399 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" podUID="1b062018-80b8-42fb-83ca-57cfb9fd805e" containerName="sequence-graph-34033" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:03:02.047227 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:02.047169 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" podUID="1b062018-80b8-42fb-83ca-57cfb9fd805e" containerName="sequence-graph-34033" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:03:04.651890 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:04.651867 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:03:04.739466 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:04.739439 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b062018-80b8-42fb-83ca-57cfb9fd805e-openshift-service-ca-bundle\") pod \"1b062018-80b8-42fb-83ca-57cfb9fd805e\" (UID: \"1b062018-80b8-42fb-83ca-57cfb9fd805e\") " Sep 16 19:03:04.739586 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:04.739473 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b062018-80b8-42fb-83ca-57cfb9fd805e-proxy-tls\") pod \"1b062018-80b8-42fb-83ca-57cfb9fd805e\" (UID: \"1b062018-80b8-42fb-83ca-57cfb9fd805e\") " Sep 16 19:03:04.739764 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:04.739743 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b062018-80b8-42fb-83ca-57cfb9fd805e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "1b062018-80b8-42fb-83ca-57cfb9fd805e" (UID: "1b062018-80b8-42fb-83ca-57cfb9fd805e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 19:03:04.741419 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:04.741397 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b062018-80b8-42fb-83ca-57cfb9fd805e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1b062018-80b8-42fb-83ca-57cfb9fd805e" (UID: "1b062018-80b8-42fb-83ca-57cfb9fd805e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 19:03:04.840771 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:04.840745 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b062018-80b8-42fb-83ca-57cfb9fd805e-openshift-service-ca-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:03:04.840771 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:04.840768 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1b062018-80b8-42fb-83ca-57cfb9fd805e-proxy-tls\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:03:05.218095 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:05.218003 2566 generic.go:356] "Generic (PLEG): container finished" podID="1b062018-80b8-42fb-83ca-57cfb9fd805e" containerID="335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515" exitCode=0 Sep 16 19:03:05.218095 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:05.218048 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" Sep 16 19:03:05.218347 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:05.218091 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" event={"ID":"1b062018-80b8-42fb-83ca-57cfb9fd805e","Type":"ContainerDied","Data":"335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515"} Sep 16 19:03:05.218347 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:05.218146 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp" event={"ID":"1b062018-80b8-42fb-83ca-57cfb9fd805e","Type":"ContainerDied","Data":"7fe6dc2500ee2055006fcd4a8db1429311752ab0c0f34b1f8cd21864b1a59f0a"} Sep 16 19:03:05.218347 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:05.218170 2566 scope.go:117] "RemoveContainer" containerID="335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515" Sep 16 19:03:05.226603 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:05.226584 2566 scope.go:117] "RemoveContainer" containerID="335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515" Sep 16 19:03:05.226867 ip-10-0-133-99 kubenswrapper[2566]: E0916 19:03:05.226841 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515\": container with ID starting with 335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515 not found: ID does not exist" containerID="335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515" Sep 16 19:03:05.226964 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:05.226870 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515"} err="failed to get container status \"335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515\": rpc error: code = NotFound desc = could not find container \"335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515\": container with ID starting with 335c01797cf6f80f9b2e20eb28d6444896b2893e7fcd2d52f602b123a89d4515 not found: ID does not exist" Sep 16 19:03:05.247297 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:05.247277 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp"] Sep 16 19:03:05.253489 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:05.253468 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-34033-5489c4744b-q5rmp"] Sep 16 19:03:06.797199 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:06.797146 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b062018-80b8-42fb-83ca-57cfb9fd805e" path="/var/lib/kubelet/pods/1b062018-80b8-42fb-83ca-57cfb9fd805e/volumes" Sep 16 19:03:34.788114 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.788081 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph"] Sep 16 19:03:34.788521 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.788410 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b062018-80b8-42fb-83ca-57cfb9fd805e" containerName="sequence-graph-34033" Sep 16 19:03:34.788521 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.788422 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b062018-80b8-42fb-83ca-57cfb9fd805e" containerName="sequence-graph-34033" Sep 16 19:03:34.788521 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.788482 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b062018-80b8-42fb-83ca-57cfb9fd805e" containerName="sequence-graph-34033" Sep 16 19:03:34.793563 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.793529 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:03:34.796603 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.796576 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-e3b29-serving-cert\"" Sep 16 19:03:34.797011 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.796994 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-e3b29-kube-rbac-proxy-sar-config\"" Sep 16 19:03:34.806070 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.806049 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph"] Sep 16 19:03:34.847387 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.847359 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5-proxy-tls\") pod \"sequence-graph-e3b29-879b5696f-mrtph\" (UID: \"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5\") " pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:03:34.847499 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.847412 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5-openshift-service-ca-bundle\") pod \"sequence-graph-e3b29-879b5696f-mrtph\" (UID: \"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5\") " pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:03:34.947971 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.947940 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5-proxy-tls\") pod \"sequence-graph-e3b29-879b5696f-mrtph\" (UID: \"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5\") " pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:03:34.948140 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.947994 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5-openshift-service-ca-bundle\") pod \"sequence-graph-e3b29-879b5696f-mrtph\" (UID: \"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5\") " pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:03:34.948754 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.948720 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5-openshift-service-ca-bundle\") pod \"sequence-graph-e3b29-879b5696f-mrtph\" (UID: \"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5\") " pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:03:34.950327 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:34.950301 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5-proxy-tls\") pod \"sequence-graph-e3b29-879b5696f-mrtph\" (UID: \"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5\") " pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:03:35.104049 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:35.103989 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:03:35.381735 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:35.381701 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph"] Sep 16 19:03:35.382620 ip-10-0-133-99 kubenswrapper[2566]: W0916 19:03:35.382595 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2476c3a5_792d_4b47_a2cf_cdaddf8d43b5.slice/crio-caa142183ddd9b4c40606c46a466a8ddbda7a05e46066891632bc8beb6ece8b4 WatchSource:0}: Error finding container caa142183ddd9b4c40606c46a466a8ddbda7a05e46066891632bc8beb6ece8b4: Status 404 returned error can't find the container with id caa142183ddd9b4c40606c46a466a8ddbda7a05e46066891632bc8beb6ece8b4 Sep 16 19:03:36.367678 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:36.367639 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" event={"ID":"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5","Type":"ContainerStarted","Data":"fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845"} Sep 16 19:03:36.368070 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:36.367692 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" event={"ID":"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5","Type":"ContainerStarted","Data":"caa142183ddd9b4c40606c46a466a8ddbda7a05e46066891632bc8beb6ece8b4"} Sep 16 19:03:36.368070 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:36.367728 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:03:36.387092 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:36.387052 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" podStartSLOduration=2.387037914 podStartE2EDuration="2.387037914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:03:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:03:36.386354406 +0000 UTC m=+1402.067209038" watchObservedRunningTime="2026-09-16 19:03:36.387037914 +0000 UTC m=+1402.067892547" Sep 16 19:03:42.379332 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:03:42.379300 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:05:14.792207 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:05:14.792158 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:05:14.797286 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:05:14.797258 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:10:14.818589 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:10:14.818558 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:10:14.823924 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:10:14.823892 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:10:59.955082 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:10:59.955045 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz"] Sep 16 19:10:59.955689 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:10:59.955257 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" podUID="0207330b-1a61-406e-be72-947ec2a29d0c" containerName="ensemble-graph-e4920" containerID="cri-o://b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33" gracePeriod=30 Sep 16 19:11:03.144572 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:03.144535 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" podUID="0207330b-1a61-406e-be72-947ec2a29d0c" containerName="ensemble-graph-e4920" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:11:08.143917 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:08.143875 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" podUID="0207330b-1a61-406e-be72-947ec2a29d0c" containerName="ensemble-graph-e4920" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:11:13.144397 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:13.144357 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" podUID="0207330b-1a61-406e-be72-947ec2a29d0c" containerName="ensemble-graph-e4920" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:11:13.144806 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:13.144531 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:11:18.144373 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:18.144336 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" podUID="0207330b-1a61-406e-be72-947ec2a29d0c" containerName="ensemble-graph-e4920" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:11:23.143830 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:23.143791 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" podUID="0207330b-1a61-406e-be72-947ec2a29d0c" containerName="ensemble-graph-e4920" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:11:28.144689 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:28.144646 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" podUID="0207330b-1a61-406e-be72-947ec2a29d0c" containerName="ensemble-graph-e4920" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:11:30.106428 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.106394 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:11:30.140712 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.140682 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0207330b-1a61-406e-be72-947ec2a29d0c-proxy-tls\") pod \"0207330b-1a61-406e-be72-947ec2a29d0c\" (UID: \"0207330b-1a61-406e-be72-947ec2a29d0c\") " Sep 16 19:11:30.140848 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.140723 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0207330b-1a61-406e-be72-947ec2a29d0c-openshift-service-ca-bundle\") pod \"0207330b-1a61-406e-be72-947ec2a29d0c\" (UID: \"0207330b-1a61-406e-be72-947ec2a29d0c\") " Sep 16 19:11:30.141084 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.141054 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0207330b-1a61-406e-be72-947ec2a29d0c-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "0207330b-1a61-406e-be72-947ec2a29d0c" (UID: "0207330b-1a61-406e-be72-947ec2a29d0c"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 19:11:30.142579 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.142559 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0207330b-1a61-406e-be72-947ec2a29d0c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0207330b-1a61-406e-be72-947ec2a29d0c" (UID: "0207330b-1a61-406e-be72-947ec2a29d0c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 19:11:30.242061 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.242008 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0207330b-1a61-406e-be72-947ec2a29d0c-proxy-tls\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:11:30.242061 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.242030 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0207330b-1a61-406e-be72-947ec2a29d0c-openshift-service-ca-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:11:30.564683 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.564652 2566 generic.go:356] "Generic (PLEG): container finished" podID="0207330b-1a61-406e-be72-947ec2a29d0c" containerID="b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33" exitCode=0 Sep 16 19:11:30.564824 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.564692 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" Sep 16 19:11:30.564824 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.564734 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" event={"ID":"0207330b-1a61-406e-be72-947ec2a29d0c","Type":"ContainerDied","Data":"b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33"} Sep 16 19:11:30.564824 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.564781 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz" event={"ID":"0207330b-1a61-406e-be72-947ec2a29d0c","Type":"ContainerDied","Data":"5d45e33aa937b7c1c7a5dcd787c7ec2747e36ca552e05e7557bd4622ab55f47d"} Sep 16 19:11:30.564824 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.564810 2566 scope.go:117] "RemoveContainer" containerID="b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33" Sep 16 19:11:30.573700 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.573678 2566 scope.go:117] "RemoveContainer" containerID="b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33" Sep 16 19:11:30.573964 ip-10-0-133-99 kubenswrapper[2566]: E0916 19:11:30.573946 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33\": container with ID starting with b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33 not found: ID does not exist" containerID="b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33" Sep 16 19:11:30.574006 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.573971 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33"} err="failed to get container status \"b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33\": rpc error: code = NotFound desc = could not find container \"b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33\": container with ID starting with b9cc330dd6e97f6d722cf5a618c9518faf0813cc07a526cea7f0d07efaee5a33 not found: ID does not exist" Sep 16 19:11:30.595091 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.595059 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz"] Sep 16 19:11:30.606412 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.606388 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-e4920-59fb4569b6-fl2qz"] Sep 16 19:11:30.796491 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:30.796466 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0207330b-1a61-406e-be72-947ec2a29d0c" path="/var/lib/kubelet/pods/0207330b-1a61-406e-be72-947ec2a29d0c/volumes" Sep 16 19:11:49.418603 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:49.418529 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph"] Sep 16 19:11:49.419057 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:49.418716 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" podUID="2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" containerName="sequence-graph-e3b29" containerID="cri-o://fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845" gracePeriod=30 Sep 16 19:11:52.377708 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:52.377658 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" podUID="2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" containerName="sequence-graph-e3b29" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:11:57.377128 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:11:57.377092 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" podUID="2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" containerName="sequence-graph-e3b29" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:12:00.556625 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.556589 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk"] Sep 16 19:12:00.557028 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.556976 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0207330b-1a61-406e-be72-947ec2a29d0c" containerName="ensemble-graph-e4920" Sep 16 19:12:00.557028 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.556989 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0207330b-1a61-406e-be72-947ec2a29d0c" containerName="ensemble-graph-e4920" Sep 16 19:12:00.557111 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.557052 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0207330b-1a61-406e-be72-947ec2a29d0c" containerName="ensemble-graph-e4920" Sep 16 19:12:00.560127 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.560106 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:00.563066 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.563046 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-617e5-serving-cert\"" Sep 16 19:12:00.563155 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.563047 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-617e5-kube-rbac-proxy-sar-config\"" Sep 16 19:12:00.657324 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.657295 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/993d2aa6-e563-43a0-828a-c20b80515441-proxy-tls\") pod \"splitter-graph-617e5-69d5f987b8-qz8dk\" (UID: \"993d2aa6-e563-43a0-828a-c20b80515441\") " pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:00.657465 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.657419 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/993d2aa6-e563-43a0-828a-c20b80515441-openshift-service-ca-bundle\") pod \"splitter-graph-617e5-69d5f987b8-qz8dk\" (UID: \"993d2aa6-e563-43a0-828a-c20b80515441\") " pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:00.659019 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.658993 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk"] Sep 16 19:12:00.758183 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.758151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/993d2aa6-e563-43a0-828a-c20b80515441-proxy-tls\") pod \"splitter-graph-617e5-69d5f987b8-qz8dk\" (UID: \"993d2aa6-e563-43a0-828a-c20b80515441\") " pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:00.758366 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.758240 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/993d2aa6-e563-43a0-828a-c20b80515441-openshift-service-ca-bundle\") pod \"splitter-graph-617e5-69d5f987b8-qz8dk\" (UID: \"993d2aa6-e563-43a0-828a-c20b80515441\") " pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:00.758818 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.758787 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/993d2aa6-e563-43a0-828a-c20b80515441-openshift-service-ca-bundle\") pod \"splitter-graph-617e5-69d5f987b8-qz8dk\" (UID: \"993d2aa6-e563-43a0-828a-c20b80515441\") " pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:00.760623 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.760599 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/993d2aa6-e563-43a0-828a-c20b80515441-proxy-tls\") pod \"splitter-graph-617e5-69d5f987b8-qz8dk\" (UID: \"993d2aa6-e563-43a0-828a-c20b80515441\") " pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:00.871067 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:00.870994 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:01.161556 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:01.161475 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk"] Sep 16 19:12:01.164869 ip-10-0-133-99 kubenswrapper[2566]: W0916 19:12:01.164839 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993d2aa6_e563_43a0_828a_c20b80515441.slice/crio-d3fdb7dcd87eb1508f62822a0a9109d163478cf952d53624640e33a80e700589 WatchSource:0}: Error finding container d3fdb7dcd87eb1508f62822a0a9109d163478cf952d53624640e33a80e700589: Status 404 returned error can't find the container with id d3fdb7dcd87eb1508f62822a0a9109d163478cf952d53624640e33a80e700589 Sep 16 19:12:01.718082 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:01.718041 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" event={"ID":"993d2aa6-e563-43a0-828a-c20b80515441","Type":"ContainerStarted","Data":"70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e"} Sep 16 19:12:01.718484 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:01.718104 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" event={"ID":"993d2aa6-e563-43a0-828a-c20b80515441","Type":"ContainerStarted","Data":"d3fdb7dcd87eb1508f62822a0a9109d163478cf952d53624640e33a80e700589"} Sep 16 19:12:01.718484 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:01.718156 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:01.752287 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:01.752238 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" podStartSLOduration=1.752222063 podStartE2EDuration="1.752222063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:12:01.750740781 +0000 UTC m=+1907.431595412" watchObservedRunningTime="2026-09-16 19:12:01.752222063 +0000 UTC m=+1907.433076695" Sep 16 19:12:02.377143 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:02.377105 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" podUID="2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" containerName="sequence-graph-e3b29" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:12:02.377364 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:02.377222 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:12:07.377764 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:07.377725 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" podUID="2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" containerName="sequence-graph-e3b29" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:12:07.728152 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:07.728081 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:10.643173 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:10.643140 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk"] Sep 16 19:12:10.643530 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:10.643342 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" podUID="993d2aa6-e563-43a0-828a-c20b80515441" containerName="splitter-graph-617e5" containerID="cri-o://70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e" gracePeriod=30 Sep 16 19:12:12.377372 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:12.377331 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" podUID="2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" containerName="sequence-graph-e3b29" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:12:12.727013 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:12.726929 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" podUID="993d2aa6-e563-43a0-828a-c20b80515441" containerName="splitter-graph-617e5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:12:17.377347 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:17.377303 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" podUID="2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" containerName="sequence-graph-e3b29" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:12:17.727642 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:17.727554 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" podUID="993d2aa6-e563-43a0-828a-c20b80515441" containerName="splitter-graph-617e5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:12:19.563849 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.563826 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:12:19.701251 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.701133 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5-proxy-tls\") pod \"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5\" (UID: \"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5\") " Sep 16 19:12:19.701251 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.701181 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5-openshift-service-ca-bundle\") pod \"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5\" (UID: \"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5\") " Sep 16 19:12:19.701601 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.701573 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" (UID: "2476c3a5-792d-4b47-a2cf-cdaddf8d43b5"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 19:12:19.703316 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.703295 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" (UID: "2476c3a5-792d-4b47-a2cf-cdaddf8d43b5"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 19:12:19.799745 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.799704 2566 generic.go:356] "Generic (PLEG): container finished" podID="2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" containerID="fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845" exitCode=0 Sep 16 19:12:19.799911 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.799767 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" Sep 16 19:12:19.799911 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.799786 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" event={"ID":"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5","Type":"ContainerDied","Data":"fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845"} Sep 16 19:12:19.799911 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.799836 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph" event={"ID":"2476c3a5-792d-4b47-a2cf-cdaddf8d43b5","Type":"ContainerDied","Data":"caa142183ddd9b4c40606c46a466a8ddbda7a05e46066891632bc8beb6ece8b4"} Sep 16 19:12:19.799911 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.799858 2566 scope.go:117] "RemoveContainer" containerID="fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845" Sep 16 19:12:19.802555 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.802533 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5-proxy-tls\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:12:19.802622 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.802590 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5-openshift-service-ca-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:12:19.808091 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.808073 2566 scope.go:117] "RemoveContainer" containerID="fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845" Sep 16 19:12:19.808439 ip-10-0-133-99 kubenswrapper[2566]: E0916 19:12:19.808418 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845\": container with ID starting with fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845 not found: ID does not exist" containerID="fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845" Sep 16 19:12:19.808507 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.808447 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845"} err="failed to get container status \"fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845\": rpc error: code = NotFound desc = could not find container \"fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845\": container with ID starting with fdcde6c7916ca0082e957d231d70b0f0ddd2d674f9054ac6767364605f205845 not found: ID does not exist" Sep 16 19:12:19.834613 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.834586 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph"] Sep 16 19:12:19.839212 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:19.839180 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-e3b29-879b5696f-mrtph"] Sep 16 19:12:20.796288 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:20.796257 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" path="/var/lib/kubelet/pods/2476c3a5-792d-4b47-a2cf-cdaddf8d43b5/volumes" Sep 16 19:12:22.727657 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:22.727619 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" podUID="993d2aa6-e563-43a0-828a-c20b80515441" containerName="splitter-graph-617e5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:12:22.728108 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:22.727723 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:27.727155 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:27.727108 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" podUID="993d2aa6-e563-43a0-828a-c20b80515441" containerName="splitter-graph-617e5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:12:32.727011 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:32.726968 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" podUID="993d2aa6-e563-43a0-828a-c20b80515441" containerName="splitter-graph-617e5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:12:37.727678 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:37.727634 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" podUID="993d2aa6-e563-43a0-828a-c20b80515441" containerName="splitter-graph-617e5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:12:40.787319 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.787296 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:40.866948 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.866923 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/993d2aa6-e563-43a0-828a-c20b80515441-openshift-service-ca-bundle\") pod \"993d2aa6-e563-43a0-828a-c20b80515441\" (UID: \"993d2aa6-e563-43a0-828a-c20b80515441\") " Sep 16 19:12:40.867080 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.866980 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/993d2aa6-e563-43a0-828a-c20b80515441-proxy-tls\") pod \"993d2aa6-e563-43a0-828a-c20b80515441\" (UID: \"993d2aa6-e563-43a0-828a-c20b80515441\") " Sep 16 19:12:40.867326 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.867305 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/993d2aa6-e563-43a0-828a-c20b80515441-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "993d2aa6-e563-43a0-828a-c20b80515441" (UID: "993d2aa6-e563-43a0-828a-c20b80515441"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 19:12:40.868908 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.868887 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/993d2aa6-e563-43a0-828a-c20b80515441-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "993d2aa6-e563-43a0-828a-c20b80515441" (UID: "993d2aa6-e563-43a0-828a-c20b80515441"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 19:12:40.892941 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.892916 2566 generic.go:356] "Generic (PLEG): container finished" podID="993d2aa6-e563-43a0-828a-c20b80515441" containerID="70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e" exitCode=0 Sep 16 19:12:40.893047 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.892964 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" Sep 16 19:12:40.893047 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.892987 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" event={"ID":"993d2aa6-e563-43a0-828a-c20b80515441","Type":"ContainerDied","Data":"70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e"} Sep 16 19:12:40.893047 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.893024 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk" event={"ID":"993d2aa6-e563-43a0-828a-c20b80515441","Type":"ContainerDied","Data":"d3fdb7dcd87eb1508f62822a0a9109d163478cf952d53624640e33a80e700589"} Sep 16 19:12:40.893047 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.893044 2566 scope.go:117] "RemoveContainer" containerID="70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e" Sep 16 19:12:40.901636 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.901616 2566 scope.go:117] "RemoveContainer" containerID="70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e" Sep 16 19:12:40.901919 ip-10-0-133-99 kubenswrapper[2566]: E0916 19:12:40.901899 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e\": container with ID starting with 70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e not found: ID does not exist" containerID="70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e" Sep 16 19:12:40.901971 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.901926 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e"} err="failed to get container status \"70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e\": rpc error: code = NotFound desc = could not find container \"70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e\": container with ID starting with 70fa973ac6fbfe0577c1f6f2dc35e516a8b4e583f17e96a2844a508e51ad531e not found: ID does not exist" Sep 16 19:12:40.922050 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.922030 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk"] Sep 16 19:12:40.928907 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.928877 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-617e5-69d5f987b8-qz8dk"] Sep 16 19:12:40.967458 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.967438 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/993d2aa6-e563-43a0-828a-c20b80515441-openshift-service-ca-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:12:40.967538 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:40.967459 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/993d2aa6-e563-43a0-828a-c20b80515441-proxy-tls\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:12:42.796575 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:42.796541 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="993d2aa6-e563-43a0-828a-c20b80515441" path="/var/lib/kubelet/pods/993d2aa6-e563-43a0-828a-c20b80515441/volumes" Sep 16 19:12:49.878872 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.878841 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk"] Sep 16 19:12:49.879248 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.879180 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" containerName="sequence-graph-e3b29" Sep 16 19:12:49.879248 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.879208 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" containerName="sequence-graph-e3b29" Sep 16 19:12:49.879248 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.879224 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="993d2aa6-e563-43a0-828a-c20b80515441" containerName="splitter-graph-617e5" Sep 16 19:12:49.879248 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.879230 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="993d2aa6-e563-43a0-828a-c20b80515441" containerName="splitter-graph-617e5" Sep 16 19:12:49.879387 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.879298 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2476c3a5-792d-4b47-a2cf-cdaddf8d43b5" containerName="sequence-graph-e3b29" Sep 16 19:12:49.879387 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.879308 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="993d2aa6-e563-43a0-828a-c20b80515441" containerName="splitter-graph-617e5" Sep 16 19:12:49.882096 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.882079 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:12:49.886831 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.886807 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Sep 16 19:12:49.886915 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.886807 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-d2d49-serving-cert\"" Sep 16 19:12:49.886962 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.886813 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-24wmq\"" Sep 16 19:12:49.887403 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.887382 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-d2d49-kube-rbac-proxy-sar-config\"" Sep 16 19:12:49.900998 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.900974 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk"] Sep 16 19:12:49.933142 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.933121 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/48c89022-abda-4ced-8ac7-b8e81f54bc47-proxy-tls\") pod \"switch-graph-d2d49-6548fd667c-bsphk\" (UID: \"48c89022-abda-4ced-8ac7-b8e81f54bc47\") " pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:12:49.933253 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:49.933162 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48c89022-abda-4ced-8ac7-b8e81f54bc47-openshift-service-ca-bundle\") pod \"switch-graph-d2d49-6548fd667c-bsphk\" (UID: \"48c89022-abda-4ced-8ac7-b8e81f54bc47\") " pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:12:50.033528 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:50.033496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/48c89022-abda-4ced-8ac7-b8e81f54bc47-proxy-tls\") pod \"switch-graph-d2d49-6548fd667c-bsphk\" (UID: \"48c89022-abda-4ced-8ac7-b8e81f54bc47\") " pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:12:50.033669 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:50.033552 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48c89022-abda-4ced-8ac7-b8e81f54bc47-openshift-service-ca-bundle\") pod \"switch-graph-d2d49-6548fd667c-bsphk\" (UID: \"48c89022-abda-4ced-8ac7-b8e81f54bc47\") " pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:12:50.033669 ip-10-0-133-99 kubenswrapper[2566]: E0916 19:12:50.033637 2566 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-d2d49-serving-cert: secret "switch-graph-d2d49-serving-cert" not found Sep 16 19:12:50.033764 ip-10-0-133-99 kubenswrapper[2566]: E0916 19:12:50.033708 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48c89022-abda-4ced-8ac7-b8e81f54bc47-proxy-tls podName:48c89022-abda-4ced-8ac7-b8e81f54bc47 nodeName:}" failed. No retries permitted until 2026-09-16 19:12:50.533694094 +0000 UTC m=+1956.214548708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/48c89022-abda-4ced-8ac7-b8e81f54bc47-proxy-tls") pod "switch-graph-d2d49-6548fd667c-bsphk" (UID: "48c89022-abda-4ced-8ac7-b8e81f54bc47") : secret "switch-graph-d2d49-serving-cert" not found Sep 16 19:12:50.034138 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:50.034122 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48c89022-abda-4ced-8ac7-b8e81f54bc47-openshift-service-ca-bundle\") pod \"switch-graph-d2d49-6548fd667c-bsphk\" (UID: \"48c89022-abda-4ced-8ac7-b8e81f54bc47\") " pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:12:50.537691 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:50.537655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/48c89022-abda-4ced-8ac7-b8e81f54bc47-proxy-tls\") pod \"switch-graph-d2d49-6548fd667c-bsphk\" (UID: \"48c89022-abda-4ced-8ac7-b8e81f54bc47\") " pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:12:50.540157 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:50.540125 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/48c89022-abda-4ced-8ac7-b8e81f54bc47-proxy-tls\") pod \"switch-graph-d2d49-6548fd667c-bsphk\" (UID: \"48c89022-abda-4ced-8ac7-b8e81f54bc47\") " pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:12:50.791609 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:50.791546 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:12:51.076227 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:51.076182 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk"] Sep 16 19:12:51.952876 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:51.952837 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" event={"ID":"48c89022-abda-4ced-8ac7-b8e81f54bc47","Type":"ContainerStarted","Data":"a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e"} Sep 16 19:12:51.953113 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:51.952928 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:12:51.953113 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:51.952950 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" event={"ID":"48c89022-abda-4ced-8ac7-b8e81f54bc47","Type":"ContainerStarted","Data":"a6ea187a8c25669a12028af447268f83d3f896aefd084fb276adaf68fdfc22db"} Sep 16 19:12:51.976514 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:51.976465 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" podStartSLOduration=2.976449336 podStartE2EDuration="2.976449336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:12:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:12:51.974604487 +0000 UTC m=+1957.655459119" watchObservedRunningTime="2026-09-16 19:12:51.976449336 +0000 UTC m=+1957.657303967" Sep 16 19:12:57.963466 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:12:57.963430 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:13:11.161568 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.161536 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx"] Sep 16 19:13:11.164963 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.164943 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:13:11.167559 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.167538 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-0e2c5-kube-rbac-proxy-sar-config\"" Sep 16 19:13:11.167653 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.167539 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-0e2c5-serving-cert\"" Sep 16 19:13:11.178744 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.178722 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx"] Sep 16 19:13:11.286005 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.285975 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f-proxy-tls\") pod \"splitter-graph-0e2c5-5df677ff7b-cvghx\" (UID: \"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f\") " pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:13:11.286159 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.286025 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f-openshift-service-ca-bundle\") pod \"splitter-graph-0e2c5-5df677ff7b-cvghx\" (UID: \"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f\") " pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:13:11.386580 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.386551 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f-proxy-tls\") pod \"splitter-graph-0e2c5-5df677ff7b-cvghx\" (UID: \"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f\") " pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:13:11.386724 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.386591 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f-openshift-service-ca-bundle\") pod \"splitter-graph-0e2c5-5df677ff7b-cvghx\" (UID: \"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f\") " pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:13:11.387195 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.387166 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f-openshift-service-ca-bundle\") pod \"splitter-graph-0e2c5-5df677ff7b-cvghx\" (UID: \"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f\") " pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:13:11.388816 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.388791 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f-proxy-tls\") pod \"splitter-graph-0e2c5-5df677ff7b-cvghx\" (UID: \"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f\") " pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:13:11.475562 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.475498 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:13:11.752059 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:11.751987 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx"] Sep 16 19:13:11.756085 ip-10-0-133-99 kubenswrapper[2566]: W0916 19:13:11.756052 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aa32f0e_bb97_4c18_9fa1_b1395c211c9f.slice/crio-4bcb0cb1ccd8d7c5232ee9471d76a6c1846068ece4c0e6732536edfcace16e20 WatchSource:0}: Error finding container 4bcb0cb1ccd8d7c5232ee9471d76a6c1846068ece4c0e6732536edfcace16e20: Status 404 returned error can't find the container with id 4bcb0cb1ccd8d7c5232ee9471d76a6c1846068ece4c0e6732536edfcace16e20 Sep 16 19:13:12.051123 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:12.051065 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" event={"ID":"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f","Type":"ContainerStarted","Data":"d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e"} Sep 16 19:13:12.051321 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:12.051141 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" event={"ID":"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f","Type":"ContainerStarted","Data":"4bcb0cb1ccd8d7c5232ee9471d76a6c1846068ece4c0e6732536edfcace16e20"} Sep 16 19:13:12.051321 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:12.051167 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:13:12.081003 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:12.080956 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" podStartSLOduration=1.080941261 podStartE2EDuration="1.080941261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 19:13:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:13:12.07893076 +0000 UTC m=+1977.759785391" watchObservedRunningTime="2026-09-16 19:13:12.080941261 +0000 UTC m=+1977.761795893" Sep 16 19:13:18.062065 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:13:18.062032 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:15:14.845514 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:15:14.845484 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:15:14.857627 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:15:14.857596 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:20:14.877827 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:20:14.877718 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:20:14.885072 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:20:14.885048 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:21:25.689993 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:25.689958 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx"] Sep 16 19:21:25.690582 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:25.690204 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" podUID="9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" containerName="splitter-graph-0e2c5" containerID="cri-o://d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e" gracePeriod=30 Sep 16 19:21:28.060898 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:28.060846 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" podUID="9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" containerName="splitter-graph-0e2c5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:21:33.061224 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:33.060831 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" podUID="9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" containerName="splitter-graph-0e2c5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:21:38.061339 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:38.061286 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" podUID="9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" containerName="splitter-graph-0e2c5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:21:38.061764 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:38.061421 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:21:43.060865 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:43.060816 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" podUID="9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" containerName="splitter-graph-0e2c5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:21:48.060942 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:48.060897 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" podUID="9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" containerName="splitter-graph-0e2c5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:21:53.060453 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:53.060409 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" podUID="9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" containerName="splitter-graph-0e2c5" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:21:55.850571 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:55.850544 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:21:55.929101 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:55.929076 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f-openshift-service-ca-bundle\") pod \"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f\" (UID: \"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f\") " Sep 16 19:21:55.929234 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:55.929129 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f-proxy-tls\") pod \"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f\" (UID: \"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f\") " Sep 16 19:21:55.929418 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:55.929395 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" (UID: "9aa32f0e-bb97-4c18-9fa1-b1395c211c9f"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 19:21:55.931024 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:55.931000 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" (UID: "9aa32f0e-bb97-4c18-9fa1-b1395c211c9f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 19:21:56.030480 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:56.030453 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f-proxy-tls\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:21:56.030480 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:56.030480 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f-openshift-service-ca-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:21:56.440867 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:56.440785 2566 generic.go:356] "Generic (PLEG): container finished" podID="9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" containerID="d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e" exitCode=0 Sep 16 19:21:56.440867 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:56.440828 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" Sep 16 19:21:56.440867 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:56.440841 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" event={"ID":"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f","Type":"ContainerDied","Data":"d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e"} Sep 16 19:21:56.441083 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:56.440882 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx" event={"ID":"9aa32f0e-bb97-4c18-9fa1-b1395c211c9f","Type":"ContainerDied","Data":"4bcb0cb1ccd8d7c5232ee9471d76a6c1846068ece4c0e6732536edfcace16e20"} Sep 16 19:21:56.441083 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:56.440904 2566 scope.go:117] "RemoveContainer" containerID="d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e" Sep 16 19:21:56.449849 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:56.449831 2566 scope.go:117] "RemoveContainer" containerID="d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e" Sep 16 19:21:56.450142 ip-10-0-133-99 kubenswrapper[2566]: E0916 19:21:56.450121 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e\": container with ID starting with d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e not found: ID does not exist" containerID="d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e" Sep 16 19:21:56.450260 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:56.450147 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e"} err="failed to get container status \"d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e\": rpc error: code = NotFound desc = could not find container \"d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e\": container with ID starting with d60c896285553348dd9b494724368c22e624f322b79abe876650fb97b4a34a9e not found: ID does not exist" Sep 16 19:21:56.470871 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:56.470848 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx"] Sep 16 19:21:56.473852 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:56.473832 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-0e2c5-5df677ff7b-cvghx"] Sep 16 19:21:56.797266 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:21:56.797239 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" path="/var/lib/kubelet/pods/9aa32f0e-bb97-4c18-9fa1-b1395c211c9f/volumes" Sep 16 19:25:14.911181 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:25:14.911068 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:25:14.918706 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:25:14.918685 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:29:09.125616 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:09.125581 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk"] Sep 16 19:29:09.126127 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:09.125845 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" podUID="48c89022-abda-4ced-8ac7-b8e81f54bc47" containerName="switch-graph-d2d49" containerID="cri-o://a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e" gracePeriod=30 Sep 16 19:29:12.961274 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:12.961230 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" podUID="48c89022-abda-4ced-8ac7-b8e81f54bc47" containerName="switch-graph-d2d49" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:29:17.961077 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:17.961027 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" podUID="48c89022-abda-4ced-8ac7-b8e81f54bc47" containerName="switch-graph-d2d49" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:29:22.961175 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:22.961130 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" podUID="48c89022-abda-4ced-8ac7-b8e81f54bc47" containerName="switch-graph-d2d49" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:29:22.961594 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:22.961275 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:29:26.733916 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:26.733878 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:27.661726 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:27.661689 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:27.961854 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:27.961772 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" podUID="48c89022-abda-4ced-8ac7-b8e81f54bc47" containerName="switch-graph-d2d49" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:29:28.564577 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:28.564543 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:29.407070 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:29.407033 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:30.285888 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:30.285851 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:31.176480 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:31.176444 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:32.016354 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:32.016316 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:32.870734 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:32.870703 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:32.961009 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:32.960974 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" podUID="48c89022-abda-4ced-8ac7-b8e81f54bc47" containerName="switch-graph-d2d49" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:29:33.724931 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:33.724898 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:34.563976 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:34.563937 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:35.408246 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:35.408207 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:36.339630 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:36.339597 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:37.221664 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:37.221571 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:37.961815 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:37.961765 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" podUID="48c89022-abda-4ced-8ac7-b8e81f54bc47" containerName="switch-graph-d2d49" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 16 19:29:38.129671 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:38.129632 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:39.001097 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.001064 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-d2d49-6548fd667c-bsphk_48c89022-abda-4ced-8ac7-b8e81f54bc47/switch-graph-d2d49/0.log" Sep 16 19:29:39.273610 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.273589 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:29:39.413815 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.413788 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/48c89022-abda-4ced-8ac7-b8e81f54bc47-proxy-tls\") pod \"48c89022-abda-4ced-8ac7-b8e81f54bc47\" (UID: \"48c89022-abda-4ced-8ac7-b8e81f54bc47\") " Sep 16 19:29:39.413815 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.413819 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48c89022-abda-4ced-8ac7-b8e81f54bc47-openshift-service-ca-bundle\") pod \"48c89022-abda-4ced-8ac7-b8e81f54bc47\" (UID: \"48c89022-abda-4ced-8ac7-b8e81f54bc47\") " Sep 16 19:29:39.414243 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.414215 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48c89022-abda-4ced-8ac7-b8e81f54bc47-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "48c89022-abda-4ced-8ac7-b8e81f54bc47" (UID: "48c89022-abda-4ced-8ac7-b8e81f54bc47"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 16 19:29:39.415691 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.415666 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48c89022-abda-4ced-8ac7-b8e81f54bc47-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "48c89022-abda-4ced-8ac7-b8e81f54bc47" (UID: "48c89022-abda-4ced-8ac7-b8e81f54bc47"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 16 19:29:39.515253 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.515176 2566 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/48c89022-abda-4ced-8ac7-b8e81f54bc47-proxy-tls\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:29:39.515253 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.515228 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/48c89022-abda-4ced-8ac7-b8e81f54bc47-openshift-service-ca-bundle\") on node \"ip-10-0-133-99.ec2.internal\" DevicePath \"\"" Sep 16 19:29:39.522645 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.522615 2566 generic.go:356] "Generic (PLEG): container finished" podID="48c89022-abda-4ced-8ac7-b8e81f54bc47" containerID="a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e" exitCode=0 Sep 16 19:29:39.522746 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.522672 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" event={"ID":"48c89022-abda-4ced-8ac7-b8e81f54bc47","Type":"ContainerDied","Data":"a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e"} Sep 16 19:29:39.522746 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.522680 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" Sep 16 19:29:39.522746 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.522744 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk" event={"ID":"48c89022-abda-4ced-8ac7-b8e81f54bc47","Type":"ContainerDied","Data":"a6ea187a8c25669a12028af447268f83d3f896aefd084fb276adaf68fdfc22db"} Sep 16 19:29:39.522899 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.522769 2566 scope.go:117] "RemoveContainer" containerID="a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e" Sep 16 19:29:39.531174 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.531158 2566 scope.go:117] "RemoveContainer" containerID="a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e" Sep 16 19:29:39.531437 ip-10-0-133-99 kubenswrapper[2566]: E0916 19:29:39.531419 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e\": container with ID starting with a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e not found: ID does not exist" containerID="a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e" Sep 16 19:29:39.531498 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.531450 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e"} err="failed to get container status \"a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e\": rpc error: code = NotFound desc = could not find container \"a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e\": container with ID starting with a9a6de25aacdc7aab6c83fa292811892168d5117626fb9eef794c6d1297ecf2e not found: ID does not exist" Sep 16 19:29:39.549599 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.549577 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk"] Sep 16 19:29:39.557603 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:39.557584 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d2d49-6548fd667c-bsphk"] Sep 16 19:29:40.796848 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:40.796803 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48c89022-abda-4ced-8ac7-b8e81f54bc47" path="/var/lib/kubelet/pods/48c89022-abda-4ced-8ac7-b8e81f54bc47/volumes" Sep 16 19:29:44.352749 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:44.352712 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-swmrz_664c47f5-9768-4395-8396-ce9ac30421e3/global-pull-secret-syncer/0.log" Sep 16 19:29:44.477175 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:44.477147 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vxc8t_28b45ecf-82c1-4811-9f2b-e9cdf59a7816/konnectivity-agent/0.log" Sep 16 19:29:44.525983 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:44.525957 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-99.ec2.internal_d5acb9fe33c38810a5100cf8c524f161/haproxy/0.log" Sep 16 19:29:48.836856 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:48.836826 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-sw7rj_02248648-cbdf-4352-b6f7-c91ecd16ab6a/cluster-monitoring-operator/0.log" Sep 16 19:29:49.257752 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:49.257713 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-np77d_e28b89a4-0e37-40a1-95ef-ee3c9bb1e549/node-exporter/0.log" Sep 16 19:29:49.290636 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:49.290609 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-np77d_e28b89a4-0e37-40a1-95ef-ee3c9bb1e549/kube-rbac-proxy/0.log" Sep 16 19:29:49.330088 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:49.330061 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-np77d_e28b89a4-0e37-40a1-95ef-ee3c9bb1e549/init-textfile/0.log" Sep 16 19:29:51.490209 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:51.490160 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-8s942_7239f659-5512-4185-86e1-397684975a7b/networking-console-plugin/0.log" Sep 16 19:29:52.070882 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.070852 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5"] Sep 16 19:29:52.071166 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.071153 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" containerName="splitter-graph-0e2c5" Sep 16 19:29:52.071233 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.071168 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" containerName="splitter-graph-0e2c5" Sep 16 19:29:52.071233 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.071179 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48c89022-abda-4ced-8ac7-b8e81f54bc47" containerName="switch-graph-d2d49" Sep 16 19:29:52.071233 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.071197 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="48c89022-abda-4ced-8ac7-b8e81f54bc47" containerName="switch-graph-d2d49" Sep 16 19:29:52.071331 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.071263 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="48c89022-abda-4ced-8ac7-b8e81f54bc47" containerName="switch-graph-d2d49" Sep 16 19:29:52.071331 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.071270 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9aa32f0e-bb97-4c18-9fa1-b1395c211c9f" containerName="splitter-graph-0e2c5" Sep 16 19:29:52.074346 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.074327 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.076658 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.076638 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sc8ck\"/\"openshift-service-ca.crt\"" Sep 16 19:29:52.076881 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.076865 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sc8ck\"/\"kube-root-ca.crt\"" Sep 16 19:29:52.077425 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.077405 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sc8ck\"/\"default-dockercfg-zs942\"" Sep 16 19:29:52.090011 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.089989 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5"] Sep 16 19:29:52.104047 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.104029 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/1.log" Sep 16 19:29:52.109388 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.109359 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-n92m9_be5b5e3e-2d54-4755-bf1e-307ebaa84ad3/console-operator/2.log" Sep 16 19:29:52.203455 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.203430 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9xc4\" (UniqueName: \"kubernetes.io/projected/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-kube-api-access-z9xc4\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.203579 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.203466 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-proc\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.203579 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.203488 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-podres\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.203579 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.203552 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-lib-modules\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.203689 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.203598 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-sys\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.304753 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.304725 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-sys\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.304880 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.304762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9xc4\" (UniqueName: \"kubernetes.io/projected/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-kube-api-access-z9xc4\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.304880 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.304786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-proc\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.304880 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.304814 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-podres\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.304880 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.304842 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-lib-modules\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.304880 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.304864 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-sys\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.305223 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.304929 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-proc\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.305223 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.304966 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-lib-modules\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.305223 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.304968 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-podres\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.317174 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.317142 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9xc4\" (UniqueName: \"kubernetes.io/projected/1ef8dfc3-db8d-4331-aec3-25dba25bdae7-kube-api-access-z9xc4\") pod \"perf-node-gather-daemonset-ptwl5\" (UID: \"1ef8dfc3-db8d-4331-aec3-25dba25bdae7\") " pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.384150 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.384081 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:52.633732 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.633708 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6dc8ff7968-ccrk4_66c6f54c-4f86-42a7-87fe-e2e8702579ea/console/0.log" Sep 16 19:29:52.659554 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.659487 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5"] Sep 16 19:29:52.662446 ip-10-0-133-99 kubenswrapper[2566]: W0916 19:29:52.662406 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1ef8dfc3_db8d_4331_aec3_25dba25bdae7.slice/crio-d6f939b4f187c13b237e4b6d407b18ab5cb3f8dddf707bab921ad59baa1d0740 WatchSource:0}: Error finding container d6f939b4f187c13b237e4b6d407b18ab5cb3f8dddf707bab921ad59baa1d0740: Status 404 returned error can't find the container with id d6f939b4f187c13b237e4b6d407b18ab5cb3f8dddf707bab921ad59baa1d0740 Sep 16 19:29:52.696395 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:52.696374 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-44pdw_31cea14b-8e78-4949-8261-b57b97651e3b/download-server/0.log" Sep 16 19:29:53.323790 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:53.323760 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-jtvst_9c7b5a24-e9ce-4270-830a-b1a4882df402/volume-data-source-validator/0.log" Sep 16 19:29:53.587459 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:53.587381 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" event={"ID":"1ef8dfc3-db8d-4331-aec3-25dba25bdae7","Type":"ContainerStarted","Data":"912c69868c8b3558fb10bace4ddb61a952a785f761e18a5edfefeed9e51d0f8c"} Sep 16 19:29:53.587459 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:53.587442 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" event={"ID":"1ef8dfc3-db8d-4331-aec3-25dba25bdae7","Type":"ContainerStarted","Data":"d6f939b4f187c13b237e4b6d407b18ab5cb3f8dddf707bab921ad59baa1d0740"} Sep 16 19:29:53.587644 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:53.587523 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:53.607666 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:53.607612 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" podStartSLOduration=1.6075983520000001 podStartE2EDuration="1.607598352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 19:29:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 19:29:53.606693386 +0000 UTC m=+2979.287548019" watchObservedRunningTime="2026-09-16 19:29:53.607598352 +0000 UTC m=+2979.288452985" Sep 16 19:29:54.373286 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:54.373256 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fvmvk_2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46/dns/0.log" Sep 16 19:29:54.399265 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:54.399239 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fvmvk_2a6a4541-4bae-4cdd-9a65-7ad3c2b4df46/kube-rbac-proxy/0.log" Sep 16 19:29:54.574968 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:54.574940 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ldzq8_bd09345e-605b-4196-a5d3-1aa3e3010635/dns-node-resolver/0.log" Sep 16 19:29:55.151646 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:55.151611 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tjc8j_58075b65-6a9b-4ef3-9776-2cf7c3a5e15a/node-ca/0.log" Sep 16 19:29:56.048694 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:56.048660 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-54bb988d4b-wn9jr_40180743-5519-4b63-8b3b-e790ef62c2be/router/0.log" Sep 16 19:29:56.455921 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:56.455845 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rm9x6_49cff26f-250f-4a66-9156-4ca5778fa510/serve-healthcheck-canary/0.log" Sep 16 19:29:56.896486 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:56.896455 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-dz57d_3ff4c1db-9b70-4dae-977e-85493cace17a/insights-operator/0.log" Sep 16 19:29:56.897249 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:56.897227 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-dz57d_3ff4c1db-9b70-4dae-977e-85493cace17a/insights-operator/1.log" Sep 16 19:29:56.918936 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:56.918915 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5j2xj_dbf9c1a5-94c1-40f6-8788-ce631c4f2200/kube-rbac-proxy/0.log" Sep 16 19:29:56.946307 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:56.946287 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5j2xj_dbf9c1a5-94c1-40f6-8788-ce631c4f2200/exporter/0.log" Sep 16 19:29:56.971319 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:56.971300 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5j2xj_dbf9c1a5-94c1-40f6-8788-ce631c4f2200/extractor/0.log" Sep 16 19:29:59.585436 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:59.585406 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-599b79d756-jjvgv_d25dd1a1-f373-4156-9f35-8867aa912667/manager/0.log" Sep 16 19:29:59.601415 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:59.601390 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sc8ck/perf-node-gather-daemonset-ptwl5" Sep 16 19:29:59.621593 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:59.621567 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-645f9c8fd6-vbz5n_1d303272-5a5d-4fb2-b0a8-a8a4199276ff/manager/0.log" Sep 16 19:29:59.836890 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:29:59.836832 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-gkkxl_b22eef33-2325-449e-94e3-8801dca5db3d/seaweedfs/0.log" Sep 16 19:30:05.103400 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:05.103367 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-nvnqb_836dca5e-9f3b-4d19-9086-6453deee99e4/kube-storage-version-migrator-operator/1.log" Sep 16 19:30:05.104146 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:05.104128 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-nvnqb_836dca5e-9f3b-4d19-9086-6453deee99e4/kube-storage-version-migrator-operator/0.log" Sep 16 19:30:06.125537 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:06.125508 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8fkdw_0368d478-39bb-4164-8c20-bb8848762359/kube-multus-additional-cni-plugins/0.log" Sep 16 19:30:06.151806 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:06.151778 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8fkdw_0368d478-39bb-4164-8c20-bb8848762359/egress-router-binary-copy/0.log" Sep 16 19:30:06.185994 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:06.185972 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8fkdw_0368d478-39bb-4164-8c20-bb8848762359/cni-plugins/0.log" Sep 16 19:30:06.217224 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:06.217182 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8fkdw_0368d478-39bb-4164-8c20-bb8848762359/bond-cni-plugin/0.log" Sep 16 19:30:06.243230 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:06.243214 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8fkdw_0368d478-39bb-4164-8c20-bb8848762359/routeoverride-cni/0.log" Sep 16 19:30:06.269839 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:06.269822 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8fkdw_0368d478-39bb-4164-8c20-bb8848762359/whereabouts-cni-bincopy/0.log" Sep 16 19:30:06.328744 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:06.328725 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8fkdw_0368d478-39bb-4164-8c20-bb8848762359/whereabouts-cni/0.log" Sep 16 19:30:06.950560 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:06.950530 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vkgn2_a0e134d9-fa78-4ee8-8ca2-489a1e626de0/kube-multus/0.log" Sep 16 19:30:06.980427 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:06.980404 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-c2scp_b83a34db-f76e-4f19-b377-32bba0f5fa2b/network-metrics-daemon/0.log" Sep 16 19:30:07.009823 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:07.009804 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-c2scp_b83a34db-f76e-4f19-b377-32bba0f5fa2b/kube-rbac-proxy/0.log" Sep 16 19:30:07.931702 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:07.931625 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b55tw_61d20eff-c5cd-43af-aa68-7ee018467a60/ovn-controller/0.log" Sep 16 19:30:07.955108 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:07.955079 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b55tw_61d20eff-c5cd-43af-aa68-7ee018467a60/ovn-acl-logging/0.log" Sep 16 19:30:07.982309 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:07.982284 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b55tw_61d20eff-c5cd-43af-aa68-7ee018467a60/kube-rbac-proxy-node/0.log" Sep 16 19:30:08.007076 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:08.007057 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b55tw_61d20eff-c5cd-43af-aa68-7ee018467a60/kube-rbac-proxy-ovn-metrics/0.log" Sep 16 19:30:08.036929 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:08.036907 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b55tw_61d20eff-c5cd-43af-aa68-7ee018467a60/northd/0.log" Sep 16 19:30:08.069386 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:08.069365 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b55tw_61d20eff-c5cd-43af-aa68-7ee018467a60/nbdb/0.log" Sep 16 19:30:08.096497 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:08.096479 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b55tw_61d20eff-c5cd-43af-aa68-7ee018467a60/sbdb/0.log" Sep 16 19:30:08.140163 ip-10-0-133-99 kubenswrapper[2566]: I0916 19:30:08.140145 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-b55tw_61d20eff-c5cd-43af-aa68-7ee018467a60/ovnkube-controller/0.log"