Sep 09 02:23:46.693631 ip-10-0-128-153 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 09 02:23:46.693796 ip-10-0-128-153 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 09 02:23:46.693894 ip-10-0-128-153 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 09 02:23:46.694321 ip-10-0-128-153 systemd[1]: Failed to start Kubernetes Kubelet. Sep 09 02:23:56.825944 ip-10-0-128-153 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 09 02:23:56.825961 ip-10-0-128-153 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 7f3e874017be45d89a8de9817097cdd4 -- Sep 09 02:25:29.383410 ip-10-0-128-153 systemd[1]: Starting Kubernetes Kubelet... Sep 09 02:25:29.812553 ip-10-0-128-153 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 02:25:29.812553 ip-10-0-128-153 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 02:25:29.812553 ip-10-0-128-153 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 02:25:29.812553 ip-10-0-128-153 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 02:25:29.812553 ip-10-0-128-153 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 02:25:29.813589 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.813497 2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 02:25:29.818773 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818751 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 02:25:29.818773 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818768 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 02:25:29.818773 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818773 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 02:25:29.818773 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818778 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 02:25:29.818773 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818782 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818786 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818790 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818794 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818798 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818802 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818805 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818809 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818813 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818820 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818825 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818830 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818833 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818837 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818841 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818845 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818849 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818858 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818862 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 02:25:29.819071 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818865 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818869 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818873 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818877 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818880 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818884 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818888 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818892 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818896 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818900 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818904 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818908 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818912 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818916 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818921 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818926 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818931 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818935 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818940 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818944 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 02:25:29.819825 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818948 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818952 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818956 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818960 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818968 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818975 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818982 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818988 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818992 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.818997 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819001 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819006 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819010 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819014 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819018 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819023 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819027 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819031 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819036 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 02:25:29.820346 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819040 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819044 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819048 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819052 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819057 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819061 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819065 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819069 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819074 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819078 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819083 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819087 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819091 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819095 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819099 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819103 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819122 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819127 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819131 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819136 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 02:25:29.820832 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819140 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819144 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819149 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819153 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819156 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819162 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819167 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819172 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819177 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819182 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819186 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819190 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819195 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819200 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819204 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819209 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 02:25:29.821332 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.819213 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 02:25:29.822192 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822176 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 02:25:29.822192 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822191 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 02:25:29.822192 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822196 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822201 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822206 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822210 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822214 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822218 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822222 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822226 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822230 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822234 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822237 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822241 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822245 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822249 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822253 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822256 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822260 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822267 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822272 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 02:25:29.822382 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822278 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822282 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822288 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822293 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822298 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822302 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822306 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822310 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822314 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822318 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822322 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822327 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822331 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822335 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822340 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822345 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822349 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822354 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822359 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 02:25:29.823363 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822364 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822368 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822372 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822376 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822380 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822384 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822387 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822391 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822396 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822400 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822404 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822410 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822416 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822420 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822426 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822430 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822434 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822438 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822442 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822447 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 02:25:29.824210 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822451 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822455 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822459 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822463 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822467 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822472 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822476 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822480 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822485 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822489 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822494 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822498 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822502 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822506 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822509 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822514 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822518 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822522 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822526 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822530 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 02:25:29.825045 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822534 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822542 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822548 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822552 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822557 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822562 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822566 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822575 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822579 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822583 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822587 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822591 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822595 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822600 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822604 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822608 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822612 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822616 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.822620 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823239 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 02:25:29.825907 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823248 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823253 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823258 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823262 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823266 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823271 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823275 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823279 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823283 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823288 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823292 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823296 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823301 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823305 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823312 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823317 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823323 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823328 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823332 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 02:25:29.826611 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823338 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823343 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823347 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823352 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823356 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823360 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823363 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823367 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823371 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823375 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823379 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823383 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823387 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823392 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823396 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823400 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823404 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823408 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823413 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823417 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 02:25:29.827156 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823421 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823425 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823429 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823433 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823437 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823444 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823450 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823454 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823459 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823464 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823469 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823473 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823477 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823484 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823491 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823495 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823499 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823503 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823507 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 02:25:29.827653 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823511 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823515 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823519 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823523 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823527 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823532 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823536 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823540 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823544 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823548 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823552 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823556 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823560 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823564 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823568 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823572 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823576 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823580 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823584 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823588 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 02:25:29.828144 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823592 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823596 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823601 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823605 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823609 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823613 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823617 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823626 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823630 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823634 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823638 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823642 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823646 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823650 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823656 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823660 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823664 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823669 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823674 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.823678 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 02:25:29.828634 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823780 2577 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823835 2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823845 2577 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823851 2577 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823857 2577 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823865 2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823872 2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823878 2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823883 2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823888 2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823894 2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823899 2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823904 2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823909 2577 flags.go:64] FLAG: --cgroup-root="" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823913 2577 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823918 2577 flags.go:64] FLAG: --client-ca-file="" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823923 2577 flags.go:64] FLAG: --cloud-provider="external" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823928 2577 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823935 2577 flags.go:64] FLAG: --cluster-domain="" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823939 2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823945 2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823950 2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823956 2577 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823962 2577 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 02:25:29.829171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823966 2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823971 2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823976 2577 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823981 2577 flags.go:64] FLAG: --contention-profiling="false" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823986 2577 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823991 2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.823996 2577 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824001 2577 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824007 2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824012 2577 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824016 2577 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824021 2577 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824026 2577 flags.go:64] FLAG: --enable-server="true" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824032 2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824039 2577 flags.go:64] FLAG: --event-burst="100" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824044 2577 flags.go:64] FLAG: --event-qps="50" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824048 2577 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824053 2577 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824057 2577 flags.go:64] FLAG: --eviction-hard="" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824063 2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824068 2577 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824072 2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824077 2577 flags.go:64] FLAG: --eviction-soft="" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824082 2577 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824087 2577 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 02:25:29.829751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824092 2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824097 2577 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824101 2577 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824120 2577 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824126 2577 flags.go:64] FLAG: --feature-gates="" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824132 2577 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824137 2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824142 2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824147 2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824152 2577 flags.go:64] FLAG: --healthz-port="10248" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824157 2577 flags.go:64] FLAG: --help="false" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824162 2577 flags.go:64] FLAG: --hostname-override="ip-10-0-128-153.ec2.internal" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824167 2577 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824172 2577 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824177 2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824182 2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824188 2577 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824193 2577 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824197 2577 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824202 2577 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824209 2577 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824213 2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824218 2577 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824223 2577 flags.go:64] FLAG: --kube-reserved="" Sep 09 02:25:29.830388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824227 2577 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824231 2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824237 2577 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824242 2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824247 2577 flags.go:64] FLAG: --lock-file="" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824251 2577 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824256 2577 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824261 2577 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824270 2577 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824274 2577 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824279 2577 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824284 2577 flags.go:64] FLAG: --logging-format="text" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824288 2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824295 2577 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824300 2577 flags.go:64] FLAG: --manifest-url="" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824305 2577 flags.go:64] FLAG: --manifest-url-header="" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824312 2577 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824317 2577 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824323 2577 flags.go:64] FLAG: --max-pods="110" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824331 2577 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824336 2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824341 2577 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824346 2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824351 2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824355 2577 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 02:25:29.830975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824360 2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824371 2577 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824376 2577 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824383 2577 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824387 2577 flags.go:64] FLAG: --pod-cidr="" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824392 2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824401 2577 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824406 2577 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824411 2577 flags.go:64] FLAG: --pods-per-core="0" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824416 2577 flags.go:64] FLAG: --port="10250" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824420 2577 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824425 2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a5c9d9d1512061ca" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824430 2577 flags.go:64] FLAG: --qos-reserved="" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824435 2577 flags.go:64] FLAG: --read-only-port="10255" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824440 2577 flags.go:64] FLAG: --register-node="true" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824445 2577 flags.go:64] FLAG: --register-with-taints="" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824450 2577 flags.go:64] FLAG: --registry-burst="10" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824455 2577 flags.go:64] FLAG: --registry-qps="5" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824460 2577 flags.go:64] FLAG: --reserved-cpus="" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824465 2577 flags.go:64] FLAG: --reserved-memory="" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824471 2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824476 2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824481 2577 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824486 2577 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824491 2577 flags.go:64] FLAG: --runonce="false" Sep 09 02:25:29.831672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824495 2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824500 2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824507 2577 flags.go:64] FLAG: --seccomp-default="false" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824511 2577 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824516 2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824521 2577 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824525 2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824530 2577 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824535 2577 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824540 2577 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824547 2577 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824551 2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824556 2577 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824561 2577 flags.go:64] FLAG: --system-cgroups="" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824566 2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824574 2577 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824580 2577 flags.go:64] FLAG: --tls-cert-file="" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824584 2577 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824591 2577 flags.go:64] FLAG: --tls-min-version="" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824596 2577 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824601 2577 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824605 2577 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824610 2577 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824615 2577 flags.go:64] FLAG: --v="2" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824621 2577 flags.go:64] FLAG: --version="false" Sep 09 02:25:29.832318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824627 2577 flags.go:64] FLAG: --vmodule="" Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824643 2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.824648 2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825269 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825290 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825296 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825301 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825306 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825311 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825319 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825324 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825329 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825333 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825338 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825342 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825347 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825352 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825366 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825372 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825377 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 02:25:29.832955 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825382 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825388 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825392 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825398 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825402 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825407 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825412 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825417 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825423 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825434 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825439 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825444 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825449 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825453 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825457 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825462 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825466 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825472 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825477 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 02:25:29.833483 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825483 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825489 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825500 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825504 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825509 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825513 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825518 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825522 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825527 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825531 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825538 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825542 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825547 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825552 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825556 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825566 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825570 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825575 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825579 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825583 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 02:25:29.833948 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825588 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825593 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825597 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825602 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825606 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825611 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825616 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825624 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825629 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825634 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825638 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825644 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825648 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825652 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825659 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825663 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825667 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825672 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825676 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825681 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 02:25:29.834460 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825690 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825695 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825701 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825705 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825709 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825714 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825718 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825722 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825727 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825731 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825736 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825741 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825745 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825754 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825759 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825764 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825769 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825773 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825777 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825782 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 02:25:29.834949 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825786 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825791 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.825795 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.826666 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.834506 2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.834524 2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.834538 2577 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.834544 2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834600 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834604 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834608 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834610 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834614 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834617 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834620 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 02:25:29.835458 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834623 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834625 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834628 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834631 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834633 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834636 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834639 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834641 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834644 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834646 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834649 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834652 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834654 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834657 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834660 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834662 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834666 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834669 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834672 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834674 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 02:25:29.835839 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834677 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834681 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834685 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834688 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834691 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834694 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834697 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834700 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834703 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834706 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834709 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834711 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834714 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834717 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834720 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834722 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834725 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834728 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834732 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 02:25:29.836396 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834734 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834737 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834740 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834742 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834745 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834747 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834750 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834754 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834757 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834761 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834764 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834767 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834770 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834772 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834775 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834778 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834781 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834784 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834786 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834789 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 02:25:29.836849 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834791 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834794 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834797 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834799 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834802 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834804 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834807 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834809 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834812 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834814 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834817 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834819 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834822 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834825 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834827 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834830 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834832 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834835 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834838 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834840 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 02:25:29.837340 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834843 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834845 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834849 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834852 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834854 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834857 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834859 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834863 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834866 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834869 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834871 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834874 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834876 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.834881 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834990 2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 02:25:29.837829 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834994 2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.834997 2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835000 2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835003 2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835006 2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835009 2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835030 2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835034 2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835037 2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835040 2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835043 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835047 2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835050 2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835053 2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835055 2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835058 2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835061 2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835063 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835066 2577 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835069 2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 02:25:29.838214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835072 2577 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835075 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835077 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835080 2577 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835082 2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835085 2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835088 2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835090 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835093 2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835096 2577 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835098 2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835101 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835103 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835120 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835124 2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835127 2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835131 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835134 2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835136 2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 02:25:29.838712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835140 2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835142 2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835144 2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835147 2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835150 2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835152 2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835155 2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835158 2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835162 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835166 2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835170 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835173 2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835176 2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835179 2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835182 2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835185 2577 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835188 2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835192 2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 02:25:29.839308 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835196 2577 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835198 2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835201 2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835203 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835206 2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835209 2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835211 2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835214 2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835216 2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835219 2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835222 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835224 2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835227 2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835229 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835232 2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835234 2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835237 2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835239 2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835242 2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835244 2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 02:25:29.839752 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835247 2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835249 2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835252 2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835254 2577 feature_gate.go:328] unrecognized feature gate: Example Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835257 2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835259 2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835262 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835265 2577 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835267 2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835270 2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835272 2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835275 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835278 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835280 2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835283 2577 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835285 2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835288 2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835290 2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835293 2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835295 2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 02:25:29.840270 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:29.835298 2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 02:25:29.840738 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.835302 2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 02:25:29.840738 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.835400 2577 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 02:25:29.840738 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.838617 2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 02:25:29.840738 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.839705 2577 server.go:1019] "Starting client certificate rotation" Sep 09 02:25:29.840738 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.839806 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 02:25:29.840738 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.839875 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 02:25:29.842550 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.842537 2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 02:25:29.843686 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.843669 2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 02:25:29.855977 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.855958 2577 log.go:25] "Validated CRI v1 runtime API" Sep 09 02:25:29.859291 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.859276 2577 log.go:25] "Validated CRI v1 image API" Sep 09 02:25:29.860770 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.860750 2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 02:25:29.864201 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.864182 2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 95d994fa-8844-4487-8e4c-a7979c54dd71:/dev/nvme0n1p4 cdcb9654-e2d3-48e3-bad6-e8061a054918:/dev/nvme0n1p3] Sep 09 02:25:29.864269 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.864200 2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 02:25:29.869332 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.869221 2577 manager.go:216] Machine: {Timestamp:2026-09-09 02:25:29.867787686 +0000 UTC m=+0.377876966 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3083633 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21ea6f24ac02bf35a53a1469dc6c89 SystemUUID:ec21ea6f-24ac-02bf-35a5-3a1469dc6c89 BootID:7f3e8740-17be-45d8-9a8d-e9817097cdd4 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e0:32:6e:ab:cb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e0:32:6e:ab:cb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:69:67:a9:3a:ce Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 02:25:29.869332 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.869328 2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 02:25:29.869442 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.869408 2577 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 02:25:29.872841 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.872817 2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 02:25:29.872986 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.872847 2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-153.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 02:25:29.873812 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.873797 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 02:25:29.874406 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.874394 2577 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 02:25:29.874442 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.874412 2577 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 02:25:29.874442 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.874420 2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 02:25:29.875152 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.875141 2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 02:25:29.877037 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.877024 2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 02:25:29.881755 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.881744 2577 state_mem.go:36] "Initialized new in-memory state store" Sep 09 02:25:29.881871 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.881862 2577 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 02:25:29.881909 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.881894 2577 kubelet.go:486] "Attempting to sync node with API server" Sep 09 02:25:29.881909 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.881906 2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 02:25:29.881963 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.881920 2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 02:25:29.881963 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.881931 2577 kubelet.go:398] "Adding apiserver pod source" Sep 09 02:25:29.881963 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.881940 2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 02:25:29.882325 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.882313 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 02:25:29.882325 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.882332 2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 02:25:29.884940 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.884922 2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 02:25:29.887741 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.887725 2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 02:25:29.887839 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.887764 2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 02:25:29.887982 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.887969 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 02:25:29.888041 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.887989 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 02:25:29.888041 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.887999 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 02:25:29.888041 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.888008 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 02:25:29.888041 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.888020 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 02:25:29.888041 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.888029 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 02:25:29.888609 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.888596 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 02:25:29.888609 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.888610 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 02:25:29.888721 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.888621 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 02:25:29.888721 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.888629 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 02:25:29.888721 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.888642 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 02:25:29.888721 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.888655 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 02:25:29.889609 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.889596 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 02:25:29.889609 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.889609 2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 02:25:29.893550 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.893535 2577 server.go:1268] "Started kubelet" Sep 09 02:25:29.893699 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.893673 2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 02:25:29.894192 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:29.894165 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-153.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 02:25:29.894285 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.894232 2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-153.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 02:25:29.894334 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:29.894284 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 02:25:29.894403 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.894342 2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 02:25:29.894451 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.894424 2577 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 02:25:29.894464 ip-10-0-128-153 systemd[1]: Started Kubernetes Kubelet. Sep 09 02:25:29.895878 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.895860 2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 02:25:29.898209 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.898193 2577 server.go:310] "Adding debug handlers to kubelet server" Sep 09 02:25:29.901573 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:29.898194 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-153.ec2.internal.18d385ecfca2960d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-153.ec2.internal,UID:ip-10-0-128-153.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-153.ec2.internal,},FirstTimestamp:2026-09-09 02:25:29.893508621 +0000 UTC m=+0.403597900,LastTimestamp:2026-09-09 02:25:29.893508621 +0000 UTC m=+0.403597900,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-153.ec2.internal,}" Sep 09 02:25:29.902533 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:29.902492 2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 02:25:29.904381 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.904364 2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 02:25:29.904473 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.904385 2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 02:25:29.904528 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:29.904491 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:29.905167 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905147 2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 02:25:29.905258 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905149 2577 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 02:25:29.905258 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905191 2577 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 02:25:29.905258 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905252 2577 reconstruct.go:97] "Volume reconstruction finished" Sep 09 02:25:29.905396 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905260 2577 reconciler.go:29] "Reconciler: start to sync state" Sep 09 02:25:29.905396 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905299 2577 factory.go:55] Registering systemd factory Sep 09 02:25:29.905396 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905314 2577 factory.go:223] Registration of the systemd container factory successfully Sep 09 02:25:29.905516 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905485 2577 factory.go:153] Registering CRI-O factory Sep 09 02:25:29.905516 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905496 2577 factory.go:223] Registration of the crio container factory successfully Sep 09 02:25:29.905609 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905546 2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 02:25:29.905609 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905563 2577 factory.go:103] Registering Raw factory Sep 09 02:25:29.905609 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905577 2577 manager.go:1196] Started watching for new ooms in manager Sep 09 02:25:29.905953 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.905937 2577 manager.go:349] Starting recovery of all containers Sep 09 02:25:29.918025 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.917878 2577 manager.go:354] Recovery completed Sep 09 02:25:29.918634 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:29.918600 2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 02:25:29.918725 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:29.918652 2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-153.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 02:25:29.922573 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.922550 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hq8j5" Sep 09 02:25:29.922771 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.922759 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:25:29.924780 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.924763 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:25:29.924860 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.924796 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:25:29.924860 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.924811 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:25:29.925511 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.925496 2577 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 02:25:29.925511 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.925512 2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 02:25:29.925628 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.925528 2577 state_mem.go:36] "Initialized new in-memory state store" Sep 09 02:25:29.925628 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.925549 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:25:29.926574 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.926558 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:25:29.926647 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.926589 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:25:29.926647 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.926602 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:25:29.927733 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.927720 2577 policy_none.go:49] "None policy: Start" Sep 09 02:25:29.927786 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.927740 2577 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 02:25:29.927786 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.927754 2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 02:25:29.928046 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.928031 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hq8j5" Sep 09 02:25:29.928500 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:29.928432 2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-153.ec2.internal.18d385ecfe7fbd53 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-153.ec2.internal,UID:ip-10-0-128-153.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-153.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-153.ec2.internal,},FirstTimestamp:2026-09-09 02:25:29.924779347 +0000 UTC m=+0.434868629,LastTimestamp:2026-09-09 02:25:29.924779347 +0000 UTC m=+0.434868629,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-153.ec2.internal,}" Sep 09 02:25:29.929802 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.929789 2577 policy_none.go:47] "Start" Sep 09 02:25:29.962698 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.962679 2577 manager.go:337] "Starting Device Plugin manager" Sep 09 02:25:29.986123 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:29.962723 2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 02:25:29.986123 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.962736 2577 server.go:87] "Starting device plugin registration server" Sep 09 02:25:29.986123 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.963023 2577 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 02:25:29.986123 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.963033 2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 02:25:29.986123 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.963098 2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 02:25:29.986123 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.963193 2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 02:25:29.986123 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:29.963201 2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 02:25:29.986123 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:29.963804 2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 02:25:29.986123 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:29.963857 2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:30.006023 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.005987 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 02:25:30.007222 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.007203 2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 02:25:30.007314 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.007261 2577 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 02:25:30.007314 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.007291 2577 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 02:25:30.007411 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:30.007383 2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 02:25:30.009503 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.009481 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 02:25:30.063462 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.063350 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:25:30.064924 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.064906 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:25:30.065028 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.064938 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:25:30.065028 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.064950 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:25:30.065028 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.064980 2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.076348 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.076325 2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.076348 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:30.076349 2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-153.ec2.internal\": node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:30.094055 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:30.094032 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:30.108352 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.108327 2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-153.ec2.internal"] Sep 09 02:25:30.108465 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.108410 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:25:30.109864 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.109850 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:25:30.109963 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.109882 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:25:30.109963 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.109900 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:25:30.112218 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.112204 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:25:30.112359 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.112345 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.112404 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.112375 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:25:30.113248 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.113228 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:25:30.113324 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.113253 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:25:30.113324 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.113264 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:25:30.113324 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.113287 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:25:30.113324 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.113307 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:25:30.113324 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.113318 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:25:30.115388 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.115373 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.115454 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.115399 2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 02:25:30.116223 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.116204 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientMemory" Sep 09 02:25:30.116326 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.116236 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 02:25:30.116326 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.116250 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeHasSufficientPID" Sep 09 02:25:30.142847 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:30.142824 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-153.ec2.internal\" not found" node="ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.147323 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:30.147307 2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-153.ec2.internal\" not found" node="ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.194365 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:30.194335 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:30.206682 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.206651 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3d139856900857b7f5ede0c0ce669dfe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal\" (UID: \"3d139856900857b7f5ede0c0ce669dfe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.206682 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.206682 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d139856900857b7f5ede0c0ce669dfe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal\" (UID: \"3d139856900857b7f5ede0c0ce669dfe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.206809 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.206698 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3788e9231500bda50f3f039c32df10fd-config\") pod \"kube-apiserver-proxy-ip-10-0-128-153.ec2.internal\" (UID: \"3788e9231500bda50f3f039c32df10fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.294863 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:30.294834 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:30.307211 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.307186 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3d139856900857b7f5ede0c0ce669dfe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal\" (UID: \"3d139856900857b7f5ede0c0ce669dfe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.307277 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.307255 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d139856900857b7f5ede0c0ce669dfe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal\" (UID: \"3d139856900857b7f5ede0c0ce669dfe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.307311 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.307274 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3788e9231500bda50f3f039c32df10fd-config\") pod \"kube-apiserver-proxy-ip-10-0-128-153.ec2.internal\" (UID: \"3788e9231500bda50f3f039c32df10fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.307311 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.307283 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3d139856900857b7f5ede0c0ce669dfe-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal\" (UID: \"3d139856900857b7f5ede0c0ce669dfe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.307311 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.307309 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3788e9231500bda50f3f039c32df10fd-config\") pod \"kube-apiserver-proxy-ip-10-0-128-153.ec2.internal\" (UID: \"3788e9231500bda50f3f039c32df10fd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.307402 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.307334 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3d139856900857b7f5ede0c0ce669dfe-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal\" (UID: \"3d139856900857b7f5ede0c0ce669dfe\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.395696 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:30.395617 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:30.445183 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.445160 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.449968 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:30.449951 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-153.ec2.internal" Sep 09 02:25:30.496577 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:30.496548 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:32.164700 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:32.164661 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:32.265230 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:32.265146 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:32.365844 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:32.365815 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:32.406449 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:32.406425 2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 02:25:32.406998 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:32.406529 2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 02:25:32.466328 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:32.466296 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:32.477691 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:32.477667 2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 02:25:32.498724 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:32.498675 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 02:20:31 +0000 UTC" deadline="2028-02-17 19:08:38.272195617 +0000 UTC" Sep 09 02:25:32.498724 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:32.498723 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12640h43m5.773477401s" Sep 09 02:25:32.567024 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:32.566992 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:32.667639 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:32.667593 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:32.680018 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:32.679989 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3788e9231500bda50f3f039c32df10fd.slice/crio-e5ec051349151244d977a76732c4615c0ab5fb4b9989e09b59558413fb5b348e WatchSource:0}: Error finding container e5ec051349151244d977a76732c4615c0ab5fb4b9989e09b59558413fb5b348e: Status 404 returned error can't find the container with id e5ec051349151244d977a76732c4615c0ab5fb4b9989e09b59558413fb5b348e Sep 09 02:25:32.680214 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:32.680202 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d139856900857b7f5ede0c0ce669dfe.slice/crio-8c8e2c3842c1b46982daa878c4ccb690878bce62f9d52bb8dc4fd4de91edcfb1 WatchSource:0}: Error finding container 8c8e2c3842c1b46982daa878c4ccb690878bce62f9d52bb8dc4fd4de91edcfb1: Status 404 returned error can't find the container with id 8c8e2c3842c1b46982daa878c4ccb690878bce62f9d52bb8dc4fd4de91edcfb1 Sep 09 02:25:32.684656 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:32.684621 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 02:25:32.767776 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:32.767742 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:32.868328 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:32.868238 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:32.969112 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:32.969073 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:33.035169 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.035140 2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 02:25:33.070159 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.070124 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:33.170773 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.170694 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:33.271272 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.271236 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:33.371727 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.371689 2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-153.ec2.internal\" not found" Sep 09 02:25:33.411252 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.411222 2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 02:25:33.450952 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.450883 2577 apiserver.go:52] "Watching apiserver" Sep 09 02:25:33.459221 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.459199 2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 02:25:33.459599 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.459576 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-blm2g","openshift-ovn-kubernetes/ovnkube-node-78gs8","kube-system/konnectivity-agent-smfw9","openshift-cluster-node-tuning-operator/tuned-lpvvz","openshift-image-registry/node-ca-t4fn5","openshift-multus/multus-4cghs","openshift-multus/multus-additional-cni-plugins-97k2r","openshift-multus/network-metrics-daemon-szmkq","openshift-network-diagnostics/network-check-target-4w7rt","kube-system/global-pull-secret-syncer-rczng","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx","openshift-dns/node-resolver-sffcl"] Sep 09 02:25:33.463560 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.463389 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-blm2g" Sep 09 02:25:33.466260 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.465816 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gcgzc\"" Sep 09 02:25:33.466260 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.466115 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 02:25:33.466260 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.466184 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 02:25:33.466731 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.466704 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.466831 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.466785 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 02:25:33.468547 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.468483 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-smfw9" Sep 09 02:25:33.469116 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.468881 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 02:25:33.469795 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.469772 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 02:25:33.469934 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.469916 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 02:25:33.470004 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.469967 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 02:25:33.470122 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.470058 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 02:25:33.470244 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.470155 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 02:25:33.470244 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.470231 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cz8cv\"" Sep 09 02:25:33.471016 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.470999 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gmlff\"" Sep 09 02:25:33.471132 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.471113 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 02:25:33.471344 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.471309 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 02:25:33.472043 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.472026 2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 02:25:33.472043 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.472038 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-153.ec2.internal" Sep 09 02:25:33.472807 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.472493 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t4fn5" Sep 09 02:25:33.475043 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.474954 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 02:25:33.475169 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.475109 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 02:25:33.475169 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.475158 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 02:25:33.475283 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.475197 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-752n6\"" Sep 09 02:25:33.475739 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.475413 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.477907 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.477845 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 02:25:33.477995 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.477970 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 02:25:33.477995 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.477990 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 02:25:33.478103 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.478044 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 02:25:33.478103 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.478055 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pqtjm\"" Sep 09 02:25:33.478562 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.478325 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.480265 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.480227 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 02:25:33.480353 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.480290 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 02:25:33.480694 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.480564 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2t5sf\"" Sep 09 02:25:33.480811 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.480772 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:33.480871 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.480838 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:33.481184 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.481163 2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 02:25:33.482057 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482029 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-run-k8s-cni-cncf-io\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.482149 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482074 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-var-lib-cni-multus\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.482149 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482103 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cc88s\" (UniqueName: \"kubernetes.io/projected/b24f78fe-031a-4be0-b88f-247678a0b92d-kube-api-access-cc88s\") pod \"iptables-alerter-blm2g\" (UID: \"b24f78fe-031a-4be0-b88f-247678a0b92d\") " pod="openshift-network-operator/iptables-alerter-blm2g" Sep 09 02:25:33.482149 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482130 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-kubelet\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.482298 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482158 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-var-lib-openvswitch\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.482298 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482186 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-run-openvswitch\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.482298 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482213 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/953c8a38-a75f-4767-9866-68b3336a65fb-agent-certs\") pod \"konnectivity-agent-smfw9\" (UID: \"953c8a38-a75f-4767-9866-68b3336a65fb\") " pod="kube-system/konnectivity-agent-smfw9" Sep 09 02:25:33.482298 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482239 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qbln\" (UniqueName: \"kubernetes.io/projected/2f661248-5853-4ec4-99ef-f03ae698507b-kube-api-access-4qbln\") pod \"node-ca-t4fn5\" (UID: \"2f661248-5853-4ec4-99ef-f03ae698507b\") " pod="openshift-image-registry/node-ca-t4fn5" Sep 09 02:25:33.482298 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482266 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-env-overrides\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.482519 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482295 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f661248-5853-4ec4-99ef-f03ae698507b-host\") pod \"node-ca-t4fn5\" (UID: \"2f661248-5853-4ec4-99ef-f03ae698507b\") " pod="openshift-image-registry/node-ca-t4fn5" Sep 09 02:25:33.482519 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482411 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-system-cni-dir\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.482519 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482435 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-multus-cni-dir\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.482674 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482505 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-var-lib-kubelet\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.482674 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482547 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-hostroot\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.482674 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482570 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-node-log\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.482674 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482573 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 02:25:33.482674 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482604 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-multus-conf-dir\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.482674 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482660 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-run-ovn\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.482996 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482681 2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" Sep 09 02:25:33.482996 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482683 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-cni-netd\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.482996 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482726 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-ovn-node-metrics-cert\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.482996 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482750 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-ovnkube-script-lib\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.482996 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482772 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/953c8a38-a75f-4767-9866-68b3336a65fb-konnectivity-ca\") pod \"konnectivity-agent-smfw9\" (UID: \"953c8a38-a75f-4767-9866-68b3336a65fb\") " pod="kube-system/konnectivity-agent-smfw9" Sep 09 02:25:33.482996 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482810 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/454c2cf7-2e0e-4f6a-a40e-33965988ce44-multus-daemon-config\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.482996 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482855 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-run-multus-certs\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.482996 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482893 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-systemd-units\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.482996 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482916 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-etc-openvswitch\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.482996 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482940 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-run-ovn-kubernetes\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.482996 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.482989 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkq9t\" (UniqueName: \"kubernetes.io/projected/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-kube-api-access-kkq9t\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483039 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-run-netns\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483065 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-etc-kubernetes\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483091 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wthv9\" (UniqueName: \"kubernetes.io/projected/454c2cf7-2e0e-4f6a-a40e-33965988ce44-kube-api-access-wthv9\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483130 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-slash\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-run-systemd\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483168 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-log-socket\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483192 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b24f78fe-031a-4be0-b88f-247678a0b92d-host-slash\") pod \"iptables-alerter-blm2g\" (UID: \"b24f78fe-031a-4be0-b88f-247678a0b92d\") " pod="openshift-network-operator/iptables-alerter-blm2g" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483207 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-multus-socket-dir-parent\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483222 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b24f78fe-031a-4be0-b88f-247678a0b92d-iptables-alerter-script\") pod \"iptables-alerter-blm2g\" (UID: \"b24f78fe-031a-4be0-b88f-247678a0b92d\") " pod="openshift-network-operator/iptables-alerter-blm2g" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483236 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-run-netns\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-cni-bin\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483275 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483291 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-cnibin\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483306 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/454c2cf7-2e0e-4f6a-a40e-33965988ce44-cni-binary-copy\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483320 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-var-lib-cni-bin\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483351 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-ovnkube-config\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.483501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483369 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2f661248-5853-4ec4-99ef-f03ae698507b-serviceca\") pod \"node-ca-t4fn5\" (UID: \"2f661248-5853-4ec4-99ef-f03ae698507b\") " pod="openshift-image-registry/node-ca-t4fn5" Sep 09 02:25:33.484318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.483549 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:33.484318 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.483656 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:33.485955 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.485937 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.488250 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.488229 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-drsnw\"" Sep 09 02:25:33.488460 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.488443 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 02:25:33.488509 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.488484 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 02:25:33.488573 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.488447 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 02:25:33.488717 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.488702 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sffcl" Sep 09 02:25:33.491325 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.491291 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 02:25:33.491421 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.491351 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 02:25:33.491421 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.491315 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.491969 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.491942 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-n4m9s\"" Sep 09 02:25:33.493118 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.493103 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-153.ec2.internal"] Sep 09 02:25:33.493285 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.493211 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:33.493285 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.493275 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:33.494245 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.493848 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 02:25:33.494245 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.494049 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4n4xn\"" Sep 09 02:25:33.494245 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.494072 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 02:25:33.495551 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.495528 2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 02:25:33.495837 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.495821 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal"] Sep 09 02:25:33.560225 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.560194 2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zt9jw" Sep 09 02:25:33.567720 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.567690 2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zt9jw" Sep 09 02:25:33.574074 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.574052 2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 02:25:33.579540 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.579496 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" event={"ID":"3d139856900857b7f5ede0c0ce669dfe","Type":"ContainerStarted","Data":"8c8e2c3842c1b46982daa878c4ccb690878bce62f9d52bb8dc4fd4de91edcfb1"} Sep 09 02:25:33.580776 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.580743 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-153.ec2.internal" event={"ID":"3788e9231500bda50f3f039c32df10fd","Type":"ContainerStarted","Data":"e5ec051349151244d977a76732c4615c0ab5fb4b9989e09b59558413fb5b348e"} Sep 09 02:25:33.583991 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.583970 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-kubernetes\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.584094 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-run-netns\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.584094 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584034 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-etc-kubernetes\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.584094 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584057 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wthv9\" (UniqueName: \"kubernetes.io/projected/454c2cf7-2e0e-4f6a-a40e-33965988ce44-kube-api-access-wthv9\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.584094 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584080 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-slash\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584093 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-etc-kubernetes\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584093 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-run-netns\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584102 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-run-systemd\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584151 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-slash\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584154 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-log-socket\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584189 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-log-socket\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584189 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584223 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-run-systemd\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584224 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-multus-socket-dir-parent\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584258 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-cni-bin\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584268 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-multus-socket-dir-parent\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584283 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d907cf-31d4-42d9-b7e1-69ee4f71c967-tmp-dir\") pod \"node-resolver-sffcl\" (UID: \"d5d907cf-31d4-42d9-b7e1-69ee4f71c967\") " pod="openshift-dns/node-resolver-sffcl" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584309 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.584313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584314 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-cni-bin\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584336 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584361 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9tlh\" (UniqueName: \"kubernetes.io/projected/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-kube-api-access-d9tlh\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584385 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-var-lib-kubelet\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584409 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-host\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584434 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/454c2cf7-2e0e-4f6a-a40e-33965988ce44-cni-binary-copy\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584457 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-ovnkube-config\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584481 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-lib-modules\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584504 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-var-lib-cni-multus\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584619 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-kubelet\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584657 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-run-openvswitch\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584675 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/953c8a38-a75f-4767-9866-68b3336a65fb-agent-certs\") pod \"konnectivity-agent-smfw9\" (UID: \"953c8a38-a75f-4767-9866-68b3336a65fb\") " pod="kube-system/konnectivity-agent-smfw9" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584733 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-run-openvswitch\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584763 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-socket-dir\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584787 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-etc-selinux\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584811 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-cnibin\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584835 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f661248-5853-4ec4-99ef-f03ae698507b-host\") pod \"node-ca-t4fn5\" (UID: \"2f661248-5853-4ec4-99ef-f03ae698507b\") " pod="openshift-image-registry/node-ca-t4fn5" Sep 09 02:25:33.584908 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584857 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-hostroot\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584911 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-sysctl-d\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584939 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-multus-conf-dir\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584940 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2f661248-5853-4ec4-99ef-f03ae698507b-host\") pod \"node-ca-t4fn5\" (UID: \"2f661248-5853-4ec4-99ef-f03ae698507b\") " pod="openshift-image-registry/node-ca-t4fn5" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.584963 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-cni-netd\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585006 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-kubelet\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585089 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/454c2cf7-2e0e-4f6a-a40e-33965988ce44-cni-binary-copy\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585080 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-multus-conf-dir\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585132 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-cni-netd\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585168 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-hostroot\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585171 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-var-lib-cni-multus\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585215 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-ovnkube-script-lib\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585256 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w74rv\" (UniqueName: \"kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv\") pod \"network-check-target-4w7rt\" (UID: \"c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016\") " pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585349 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70038143-027a-442d-bc5f-c8511317e7b5-etc-tuned\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585358 2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585389 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-systemd-units\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585410 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-ovnkube-config\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585416 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-sys-fs\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.585726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585464 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-modprobe-d\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585466 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-systemd-units\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585499 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-sysctl-conf\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585525 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70038143-027a-442d-bc5f-c8511317e7b5-tmp\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585551 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b87ad709-411d-4c4c-81a7-100bf4ac44d5-kubelet-config\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585577 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b24f78fe-031a-4be0-b88f-247678a0b92d-host-slash\") pod \"iptables-alerter-blm2g\" (UID: \"b24f78fe-031a-4be0-b88f-247678a0b92d\") " pod="openshift-network-operator/iptables-alerter-blm2g" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585611 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b24f78fe-031a-4be0-b88f-247678a0b92d-iptables-alerter-script\") pod \"iptables-alerter-blm2g\" (UID: \"b24f78fe-031a-4be0-b88f-247678a0b92d\") " pod="openshift-network-operator/iptables-alerter-blm2g" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585656 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-run-netns\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585671 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b24f78fe-031a-4be0-b88f-247678a0b92d-host-slash\") pod \"iptables-alerter-blm2g\" (UID: \"b24f78fe-031a-4be0-b88f-247678a0b92d\") " pod="openshift-network-operator/iptables-alerter-blm2g" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585689 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585725 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585733 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-run-netns\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585766 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-ovnkube-script-lib\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585768 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-registration-dir\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585821 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585789 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585850 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-cnibin\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.586576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585874 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-var-lib-cni-bin\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585899 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-tuning-conf-dir\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585915 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-cnibin\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585923 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-systemd\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585945 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-run\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585959 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-var-lib-cni-bin\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585969 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2f661248-5853-4ec4-99ef-f03ae698507b-serviceca\") pod \"node-ca-t4fn5\" (UID: \"2f661248-5853-4ec4-99ef-f03ae698507b\") " pod="openshift-image-registry/node-ca-t4fn5" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.585992 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-run-k8s-cni-cncf-io\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586016 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cc88s\" (UniqueName: \"kubernetes.io/projected/b24f78fe-031a-4be0-b88f-247678a0b92d-kube-api-access-cc88s\") pod \"iptables-alerter-blm2g\" (UID: \"b24f78fe-031a-4be0-b88f-247678a0b92d\") " pod="openshift-network-operator/iptables-alerter-blm2g" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586039 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-var-lib-openvswitch\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586044 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-run-k8s-cni-cncf-io\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586064 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9crf\" (UniqueName: \"kubernetes.io/projected/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-kube-api-access-m9crf\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586088 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-system-cni-dir\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586112 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-cni-binary-copy\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586136 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg4wc\" (UniqueName: \"kubernetes.io/projected/70038143-027a-442d-bc5f-c8511317e7b5-kube-api-access-sg4wc\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586149 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b24f78fe-031a-4be0-b88f-247678a0b92d-iptables-alerter-script\") pod \"iptables-alerter-blm2g\" (UID: \"b24f78fe-031a-4be0-b88f-247678a0b92d\") " pod="openshift-network-operator/iptables-alerter-blm2g" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586161 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qbln\" (UniqueName: \"kubernetes.io/projected/2f661248-5853-4ec4-99ef-f03ae698507b-kube-api-access-4qbln\") pod \"node-ca-t4fn5\" (UID: \"2f661248-5853-4ec4-99ef-f03ae698507b\") " pod="openshift-image-registry/node-ca-t4fn5" Sep 09 02:25:33.587323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586194 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-env-overrides\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586218 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-device-dir\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586239 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-sys\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586261 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-system-cni-dir\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586281 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-multus-cni-dir\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586302 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-var-lib-kubelet\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586321 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-node-log\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586344 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5d907cf-31d4-42d9-b7e1-69ee4f71c967-hosts-file\") pod \"node-resolver-sffcl\" (UID: \"d5d907cf-31d4-42d9-b7e1-69ee4f71c967\") " pod="openshift-dns/node-resolver-sffcl" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586386 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-var-lib-openvswitch\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586397 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2f661248-5853-4ec4-99ef-f03ae698507b-serviceca\") pod \"node-ca-t4fn5\" (UID: \"2f661248-5853-4ec4-99ef-f03ae698507b\") " pod="openshift-image-registry/node-ca-t4fn5" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586423 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-994tv\" (UniqueName: \"kubernetes.io/projected/d5d907cf-31d4-42d9-b7e1-69ee4f71c967-kube-api-access-994tv\") pod \"node-resolver-sffcl\" (UID: \"d5d907cf-31d4-42d9-b7e1-69ee4f71c967\") " pod="openshift-dns/node-resolver-sffcl" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586449 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b87ad709-411d-4c4c-81a7-100bf4ac44d5-dbus\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586454 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-system-cni-dir\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586454 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-var-lib-kubelet\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586478 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-run-ovn\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586494 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-node-log\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586522 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-multus-cni-dir\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586534 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-ovn-node-metrics-cert\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586563 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-run-ovn\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586595 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-env-overrides\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/953c8a38-a75f-4767-9866-68b3336a65fb-konnectivity-ca\") pod \"konnectivity-agent-smfw9\" (UID: \"953c8a38-a75f-4767-9866-68b3336a65fb\") " pod="kube-system/konnectivity-agent-smfw9" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586657 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/454c2cf7-2e0e-4f6a-a40e-33965988ce44-multus-daemon-config\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586693 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-run-multus-certs\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586717 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-etc-openvswitch\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586742 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-run-ovn-kubernetes\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkq9t\" (UniqueName: \"kubernetes.io/projected/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-kube-api-access-kkq9t\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586804 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwsmq\" (UniqueName: \"kubernetes.io/projected/3195ff7f-9445-49b7-a438-2fcb5667f15b-kube-api-access-rwsmq\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586830 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-sysconfig\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586920 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/454c2cf7-2e0e-4f6a-a40e-33965988ce44-host-run-multus-certs\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586960 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-host-run-ovn-kubernetes\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.586997 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-etc-openvswitch\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.587102 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/953c8a38-a75f-4767-9866-68b3336a65fb-konnectivity-ca\") pod \"konnectivity-agent-smfw9\" (UID: \"953c8a38-a75f-4767-9866-68b3336a65fb\") " pod="kube-system/konnectivity-agent-smfw9" Sep 09 02:25:33.588902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.587995 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/454c2cf7-2e0e-4f6a-a40e-33965988ce44-multus-daemon-config\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.589668 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.589586 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-ovn-node-metrics-cert\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.590033 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.590009 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/953c8a38-a75f-4767-9866-68b3336a65fb-agent-certs\") pod \"konnectivity-agent-smfw9\" (UID: \"953c8a38-a75f-4767-9866-68b3336a65fb\") " pod="kube-system/konnectivity-agent-smfw9" Sep 09 02:25:33.595678 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.595228 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wthv9\" (UniqueName: \"kubernetes.io/projected/454c2cf7-2e0e-4f6a-a40e-33965988ce44-kube-api-access-wthv9\") pod \"multus-4cghs\" (UID: \"454c2cf7-2e0e-4f6a-a40e-33965988ce44\") " pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.595678 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.595642 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cc88s\" (UniqueName: \"kubernetes.io/projected/b24f78fe-031a-4be0-b88f-247678a0b92d-kube-api-access-cc88s\") pod \"iptables-alerter-blm2g\" (UID: \"b24f78fe-031a-4be0-b88f-247678a0b92d\") " pod="openshift-network-operator/iptables-alerter-blm2g" Sep 09 02:25:33.596951 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.596907 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkq9t\" (UniqueName: \"kubernetes.io/projected/8dfc061f-fdd5-4e0d-8a05-9e3219360bd9-kube-api-access-kkq9t\") pod \"ovnkube-node-78gs8\" (UID: \"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9\") " pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.598434 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.598414 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qbln\" (UniqueName: \"kubernetes.io/projected/2f661248-5853-4ec4-99ef-f03ae698507b-kube-api-access-4qbln\") pod \"node-ca-t4fn5\" (UID: \"2f661248-5853-4ec4-99ef-f03ae698507b\") " pod="openshift-image-registry/node-ca-t4fn5" Sep 09 02:25:33.687911 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.687875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:33.687911 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.687920 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-registration-dir\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.688149 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.687940 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:33.688149 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.687967 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-tuning-conf-dir\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.688149 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.687989 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-systemd\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.688149 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688011 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-run\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.688149 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688012 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-registration-dir\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.688149 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688041 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9crf\" (UniqueName: \"kubernetes.io/projected/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-kube-api-access-m9crf\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:33.688149 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688067 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-system-cni-dir\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.688149 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.688038 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:25:33.688149 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.688114 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688154 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-system-cni-dir\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688090 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-cni-binary-copy\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688114 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-run\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.688194 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret podName:b87ad709-411d-4c4c-81a7-100bf4ac44d5 nodeName:}" failed. No retries permitted until 2026-09-09 02:25:34.188172536 +0000 UTC m=+3.130741446 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret") pod "global-pull-secret-syncer-rczng" (UID: "b87ad709-411d-4c4c-81a7-100bf4ac44d5") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688215 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-systemd\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.688233 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs podName:06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab nodeName:}" failed. No retries permitted until 2026-09-09 02:25:34.188221837 +0000 UTC m=+3.130790738 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs") pod "network-metrics-daemon-szmkq" (UID: "06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688252 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg4wc\" (UniqueName: \"kubernetes.io/projected/70038143-027a-442d-bc5f-c8511317e7b5-kube-api-access-sg4wc\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688283 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-device-dir\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688293 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-tuning-conf-dir\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688303 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-sys\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688344 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-device-dir\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688342 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5d907cf-31d4-42d9-b7e1-69ee4f71c967-hosts-file\") pod \"node-resolver-sffcl\" (UID: \"d5d907cf-31d4-42d9-b7e1-69ee4f71c967\") " pod="openshift-dns/node-resolver-sffcl" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688386 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-994tv\" (UniqueName: \"kubernetes.io/projected/d5d907cf-31d4-42d9-b7e1-69ee4f71c967-kube-api-access-994tv\") pod \"node-resolver-sffcl\" (UID: \"d5d907cf-31d4-42d9-b7e1-69ee4f71c967\") " pod="openshift-dns/node-resolver-sffcl" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688403 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-sys\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688411 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b87ad709-411d-4c4c-81a7-100bf4ac44d5-dbus\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:33.688616 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688444 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwsmq\" (UniqueName: \"kubernetes.io/projected/3195ff7f-9445-49b7-a438-2fcb5667f15b-kube-api-access-rwsmq\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688468 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-sysconfig\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688468 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d5d907cf-31d4-42d9-b7e1-69ee4f71c967-hosts-file\") pod \"node-resolver-sffcl\" (UID: \"d5d907cf-31d4-42d9-b7e1-69ee4f71c967\") " pod="openshift-dns/node-resolver-sffcl" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688490 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-kubernetes\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688505 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b87ad709-411d-4c4c-81a7-100bf4ac44d5-dbus\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688519 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688524 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-sysconfig\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688541 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-kubernetes\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688545 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d907cf-31d4-42d9-b7e1-69ee4f71c967-tmp-dir\") pod \"node-resolver-sffcl\" (UID: \"d5d907cf-31d4-42d9-b7e1-69ee4f71c967\") " pod="openshift-dns/node-resolver-sffcl" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688571 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688588 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688618 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688654 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-cni-binary-copy\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688661 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9tlh\" (UniqueName: \"kubernetes.io/projected/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-kube-api-access-d9tlh\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688686 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-var-lib-kubelet\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688735 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-var-lib-kubelet\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688799 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-host\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.689232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688832 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-lib-modules\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688846 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d907cf-31d4-42d9-b7e1-69ee4f71c967-tmp-dir\") pod \"node-resolver-sffcl\" (UID: \"d5d907cf-31d4-42d9-b7e1-69ee4f71c967\") " pod="openshift-dns/node-resolver-sffcl" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688861 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-socket-dir\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688883 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-etc-selinux\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688905 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-cnibin\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688920 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-host\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688926 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-sysctl-d\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688959 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74rv\" (UniqueName: \"kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv\") pod \"network-check-target-4w7rt\" (UID: \"c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016\") " pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688969 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-lib-modules\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688981 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70038143-027a-442d-bc5f-c8511317e7b5-etc-tuned\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689005 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-sys-fs\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689009 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-socket-dir\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689029 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-modprobe-d\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689034 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-etc-selinux\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689051 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-sysctl-conf\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.688960 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-cnibin\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689070 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.690002 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689093 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70038143-027a-442d-bc5f-c8511317e7b5-tmp\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.690812 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689120 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b87ad709-411d-4c4c-81a7-100bf4ac44d5-kubelet-config\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:33.690812 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689126 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/3195ff7f-9445-49b7-a438-2fcb5667f15b-sys-fs\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.690812 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689137 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-modprobe-d\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.690812 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689133 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.690812 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689168 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-sysctl-conf\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.690812 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689227 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b87ad709-411d-4c4c-81a7-100bf4ac44d5-kubelet-config\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:33.690812 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.689315 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/70038143-027a-442d-bc5f-c8511317e7b5-etc-sysctl-d\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.691643 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.691608 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/70038143-027a-442d-bc5f-c8511317e7b5-etc-tuned\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.691739 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.691677 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70038143-027a-442d-bc5f-c8511317e7b5-tmp\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.694799 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.694769 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:25:33.694799 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.694796 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:25:33.694954 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.694810 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w74rv for pod openshift-network-diagnostics/network-check-target-4w7rt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:25:33.694954 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:33.694873 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv podName:c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016 nodeName:}" failed. No retries permitted until 2026-09-09 02:25:34.19485582 +0000 UTC m=+3.137424716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w74rv" (UniqueName: "kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv") pod "network-check-target-4w7rt" (UID: "c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:25:33.697391 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.697338 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg4wc\" (UniqueName: \"kubernetes.io/projected/70038143-027a-442d-bc5f-c8511317e7b5-kube-api-access-sg4wc\") pod \"tuned-lpvvz\" (UID: \"70038143-027a-442d-bc5f-c8511317e7b5\") " pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.697391 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.697387 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9crf\" (UniqueName: \"kubernetes.io/projected/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-kube-api-access-m9crf\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:33.697562 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.697539 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9tlh\" (UniqueName: \"kubernetes.io/projected/b6b7de4b-5150-40f8-9ef4-f6c50994bb97-kube-api-access-d9tlh\") pod \"multus-additional-cni-plugins-97k2r\" (UID: \"b6b7de4b-5150-40f8-9ef4-f6c50994bb97\") " pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.698023 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.698006 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-994tv\" (UniqueName: \"kubernetes.io/projected/d5d907cf-31d4-42d9-b7e1-69ee4f71c967-kube-api-access-994tv\") pod \"node-resolver-sffcl\" (UID: \"d5d907cf-31d4-42d9-b7e1-69ee4f71c967\") " pod="openshift-dns/node-resolver-sffcl" Sep 09 02:25:33.698234 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.698216 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwsmq\" (UniqueName: \"kubernetes.io/projected/3195ff7f-9445-49b7-a438-2fcb5667f15b-kube-api-access-rwsmq\") pod \"aws-ebs-csi-driver-node-rpsxx\" (UID: \"3195ff7f-9445-49b7-a438-2fcb5667f15b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.780566 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.780502 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-blm2g" Sep 09 02:25:33.789375 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.789353 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:33.799121 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.799096 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-smfw9" Sep 09 02:25:33.807689 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.807671 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-t4fn5" Sep 09 02:25:33.819474 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.819451 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4cghs" Sep 09 02:25:33.828159 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.828142 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-97k2r" Sep 09 02:25:33.836675 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.836651 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" Sep 09 02:25:33.842385 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.842368 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sffcl" Sep 09 02:25:33.848762 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.848741 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" Sep 09 02:25:33.992371 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:33.992343 2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 02:25:34.193501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.193419 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:34.193501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.193455 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:34.193701 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:34.193566 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:25:34.193701 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:34.193620 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:25:34.193701 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:34.193625 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs podName:06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab nodeName:}" failed. No retries permitted until 2026-09-09 02:25:35.193609589 +0000 UTC m=+4.136178481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs") pod "network-metrics-daemon-szmkq" (UID: "06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:25:34.193701 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:34.193690 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret podName:b87ad709-411d-4c4c-81a7-100bf4ac44d5 nodeName:}" failed. No retries permitted until 2026-09-09 02:25:35.193678687 +0000 UTC m=+4.136247575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret") pod "global-pull-secret-syncer-rczng" (UID: "b87ad709-411d-4c4c-81a7-100bf4ac44d5") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:25:34.294104 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.294076 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74rv\" (UniqueName: \"kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv\") pod \"network-check-target-4w7rt\" (UID: \"c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016\") " pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:34.294276 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:34.294224 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:25:34.294276 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:34.294242 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:25:34.294276 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:34.294250 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w74rv for pod openshift-network-diagnostics/network-check-target-4w7rt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:25:34.294415 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:34.294320 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv podName:c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016 nodeName:}" failed. No retries permitted until 2026-09-09 02:25:35.294307128 +0000 UTC m=+4.236876037 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w74rv" (UniqueName: "kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv") pod "network-check-target-4w7rt" (UID: "c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:25:34.524227 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:34.524077 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953c8a38_a75f_4767_9866_68b3336a65fb.slice/crio-6289bfbc1c1ce9ee81c40726d327241d6732e6c4012f5da9d1af3a0d5f6c3bf4 WatchSource:0}: Error finding container 6289bfbc1c1ce9ee81c40726d327241d6732e6c4012f5da9d1af3a0d5f6c3bf4: Status 404 returned error can't find the container with id 6289bfbc1c1ce9ee81c40726d327241d6732e6c4012f5da9d1af3a0d5f6c3bf4 Sep 09 02:25:34.526925 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:34.526779 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d907cf_31d4_42d9_b7e1_69ee4f71c967.slice/crio-987fce11dc6a4c8418381d0d53bfe61273f315209dd31f36534a3bd19e89480d WatchSource:0}: Error finding container 987fce11dc6a4c8418381d0d53bfe61273f315209dd31f36534a3bd19e89480d: Status 404 returned error can't find the container with id 987fce11dc6a4c8418381d0d53bfe61273f315209dd31f36534a3bd19e89480d Sep 09 02:25:34.529359 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:34.529337 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f661248_5853_4ec4_99ef_f03ae698507b.slice/crio-cab622e1695ba320d262ddd07ffbef9f71465391ae68fdfe70a92885ab5f2f1a WatchSource:0}: Error finding container cab622e1695ba320d262ddd07ffbef9f71465391ae68fdfe70a92885ab5f2f1a: Status 404 returned error can't find the container with id cab622e1695ba320d262ddd07ffbef9f71465391ae68fdfe70a92885ab5f2f1a Sep 09 02:25:34.530049 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:34.530028 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod454c2cf7_2e0e_4f6a_a40e_33965988ce44.slice/crio-292acb8ca4f61839db55d4df9b448436eeb7803d99c612eab20bbad75aaf67c3 WatchSource:0}: Error finding container 292acb8ca4f61839db55d4df9b448436eeb7803d99c612eab20bbad75aaf67c3: Status 404 returned error can't find the container with id 292acb8ca4f61839db55d4df9b448436eeb7803d99c612eab20bbad75aaf67c3 Sep 09 02:25:34.531155 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:25:34.531133 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb24f78fe_031a_4be0_b88f_247678a0b92d.slice/crio-bebcbe72c2af3d19a7955096cbfa77f3ced1434a2bdd8208030e93b6634fb939 WatchSource:0}: Error finding container bebcbe72c2af3d19a7955096cbfa77f3ced1434a2bdd8208030e93b6634fb939: Status 404 returned error can't find the container with id bebcbe72c2af3d19a7955096cbfa77f3ced1434a2bdd8208030e93b6634fb939 Sep 09 02:25:34.571949 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.571906 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 02:20:33 +0000 UTC" deadline="2028-05-25 01:51:06.250740666 +0000 UTC" Sep 09 02:25:34.571949 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.571942 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14975h25m31.678803474s" Sep 09 02:25:34.575910 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.575892 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:34.576034 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:34.576015 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:34.583023 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.583002 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-smfw9" event={"ID":"953c8a38-a75f-4767-9866-68b3336a65fb","Type":"ContainerStarted","Data":"6289bfbc1c1ce9ee81c40726d327241d6732e6c4012f5da9d1af3a0d5f6c3bf4"} Sep 09 02:25:34.584048 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.584027 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97k2r" event={"ID":"b6b7de4b-5150-40f8-9ef4-f6c50994bb97","Type":"ContainerStarted","Data":"f53065ad43bda328dd6ac9e414de9db7faa56d281b2e49d0a1785f772163b1c7"} Sep 09 02:25:34.585251 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.585010 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t4fn5" event={"ID":"2f661248-5853-4ec4-99ef-f03ae698507b","Type":"ContainerStarted","Data":"cab622e1695ba320d262ddd07ffbef9f71465391ae68fdfe70a92885ab5f2f1a"} Sep 09 02:25:34.586224 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.586201 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sffcl" event={"ID":"d5d907cf-31d4-42d9-b7e1-69ee4f71c967","Type":"ContainerStarted","Data":"987fce11dc6a4c8418381d0d53bfe61273f315209dd31f36534a3bd19e89480d"} Sep 09 02:25:34.590014 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.589992 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" event={"ID":"3195ff7f-9445-49b7-a438-2fcb5667f15b","Type":"ContainerStarted","Data":"34ca1342724988d318cd4ef28c9028965ccc73ad5745088f348c909e629aea7e"} Sep 09 02:25:34.592943 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.592924 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" event={"ID":"70038143-027a-442d-bc5f-c8511317e7b5","Type":"ContainerStarted","Data":"a2f2e2b3c663857cb6004c57b619373ef5b4a07ecd02997b6bf8517de9b0f1d5"} Sep 09 02:25:34.597008 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.596989 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" event={"ID":"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9","Type":"ContainerStarted","Data":"1f2afef9819d17a400e57a056dc07585654c26d2904b2db5acf9fce77c3dce3b"} Sep 09 02:25:34.598112 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.598094 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-blm2g" event={"ID":"b24f78fe-031a-4be0-b88f-247678a0b92d","Type":"ContainerStarted","Data":"bebcbe72c2af3d19a7955096cbfa77f3ced1434a2bdd8208030e93b6634fb939"} Sep 09 02:25:34.599136 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:34.599119 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4cghs" event={"ID":"454c2cf7-2e0e-4f6a-a40e-33965988ce44","Type":"ContainerStarted","Data":"292acb8ca4f61839db55d4df9b448436eeb7803d99c612eab20bbad75aaf67c3"} Sep 09 02:25:35.200645 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:35.200546 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:35.200645 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:35.200599 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:35.200864 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:35.200738 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:25:35.200864 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:35.200785 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret podName:b87ad709-411d-4c4c-81a7-100bf4ac44d5 nodeName:}" failed. No retries permitted until 2026-09-09 02:25:37.200772419 +0000 UTC m=+6.143341312 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret") pod "global-pull-secret-syncer-rczng" (UID: "b87ad709-411d-4c4c-81a7-100bf4ac44d5") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:25:35.201158 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:35.201142 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:25:35.201232 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:35.201185 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs podName:06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab nodeName:}" failed. No retries permitted until 2026-09-09 02:25:37.201173856 +0000 UTC m=+6.143742744 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs") pod "network-metrics-daemon-szmkq" (UID: "06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:25:35.305686 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:35.301931 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74rv\" (UniqueName: \"kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv\") pod \"network-check-target-4w7rt\" (UID: \"c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016\") " pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:35.305686 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:35.302135 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:25:35.305686 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:35.302157 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:25:35.305686 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:35.302170 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w74rv for pod openshift-network-diagnostics/network-check-target-4w7rt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:25:35.305686 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:35.302228 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv podName:c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016 nodeName:}" failed. No retries permitted until 2026-09-09 02:25:37.302210239 +0000 UTC m=+6.244779149 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w74rv" (UniqueName: "kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv") pod "network-check-target-4w7rt" (UID: "c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:25:35.576719 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:35.572715 2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 02:20:33 +0000 UTC" deadline="2028-03-02 15:22:53.934554391 +0000 UTC" Sep 09 02:25:35.576719 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:35.572778 2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12972h57m18.361780103s" Sep 09 02:25:35.576719 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:35.575769 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:35.576719 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:35.575890 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:35.576719 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:35.575933 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:35.576719 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:35.576049 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:35.611299 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:35.610675 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-153.ec2.internal" event={"ID":"3788e9231500bda50f3f039c32df10fd","Type":"ContainerStarted","Data":"3079ec253efc013250dca738af44c1535a08d6d8947e80c6709bdf726cb9ca7b"} Sep 09 02:25:35.616215 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:35.616184 2577 generic.go:356] "Generic (PLEG): container finished" podID="3d139856900857b7f5ede0c0ce669dfe" containerID="ba3e0019105cd414c6450d887ad7743e2a71a2fd0e2ffc8538613ae49509ac86" exitCode=0 Sep 09 02:25:35.616345 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:35.616230 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" event={"ID":"3d139856900857b7f5ede0c0ce669dfe","Type":"ContainerDied","Data":"ba3e0019105cd414c6450d887ad7743e2a71a2fd0e2ffc8538613ae49509ac86"} Sep 09 02:25:35.624895 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:35.624849 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-153.ec2.internal" podStartSLOduration=2.624831539 podStartE2EDuration="2.624831539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:25:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:25:35.624593417 +0000 UTC m=+4.567162329" watchObservedRunningTime="2026-09-09 02:25:35.624831539 +0000 UTC m=+4.567400483" Sep 09 02:25:36.576054 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:36.575978 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:36.576207 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:36.576106 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:36.647235 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:36.647195 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" event={"ID":"3d139856900857b7f5ede0c0ce669dfe","Type":"ContainerStarted","Data":"7157f6c8f8fe8d69ea34e32d6f45bdb4d0242e9bdbaa8ee06e8737d4279a949d"} Sep 09 02:25:36.663415 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:36.663243 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-153.ec2.internal" podStartSLOduration=3.663222877 podStartE2EDuration="3.663222877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:25:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:25:36.662061805 +0000 UTC m=+5.604630719" watchObservedRunningTime="2026-09-09 02:25:36.663222877 +0000 UTC m=+5.605791788" Sep 09 02:25:37.221788 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:37.221084 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:37.221788 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:37.221131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:37.221788 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:37.221261 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:25:37.221788 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:37.221319 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret podName:b87ad709-411d-4c4c-81a7-100bf4ac44d5 nodeName:}" failed. No retries permitted until 2026-09-09 02:25:41.221301196 +0000 UTC m=+10.163870098 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret") pod "global-pull-secret-syncer-rczng" (UID: "b87ad709-411d-4c4c-81a7-100bf4ac44d5") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:25:37.221788 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:37.221718 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:25:37.221788 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:37.221760 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs podName:06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab nodeName:}" failed. No retries permitted until 2026-09-09 02:25:41.221747042 +0000 UTC m=+10.164315936 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs") pod "network-metrics-daemon-szmkq" (UID: "06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:25:37.322199 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:37.321607 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74rv\" (UniqueName: \"kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv\") pod \"network-check-target-4w7rt\" (UID: \"c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016\") " pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:37.322199 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:37.321773 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:25:37.322199 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:37.321793 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:25:37.322199 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:37.321806 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w74rv for pod openshift-network-diagnostics/network-check-target-4w7rt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:25:37.322199 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:37.321859 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv podName:c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016 nodeName:}" failed. No retries permitted until 2026-09-09 02:25:41.321840551 +0000 UTC m=+10.264409458 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w74rv" (UniqueName: "kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv") pod "network-check-target-4w7rt" (UID: "c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:25:37.577781 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:37.577697 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:37.577933 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:37.577826 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:37.578192 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:37.578173 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:37.578301 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:37.578261 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:38.576084 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:38.576025 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:38.576522 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:38.576160 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:39.575587 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:39.575553 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:39.575763 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:39.575704 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:39.575823 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:39.575759 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:39.575899 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:39.575869 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:40.575791 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:40.575755 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:40.576353 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:40.575906 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:41.256432 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:41.256393 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:41.256606 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:41.256444 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:41.256606 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:41.256515 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:25:41.256606 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:41.256593 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs podName:06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab nodeName:}" failed. No retries permitted until 2026-09-09 02:25:49.256573883 +0000 UTC m=+18.199142789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs") pod "network-metrics-daemon-szmkq" (UID: "06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:25:41.256606 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:41.256525 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:25:41.256901 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:41.256676 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret podName:b87ad709-411d-4c4c-81a7-100bf4ac44d5 nodeName:}" failed. No retries permitted until 2026-09-09 02:25:49.25665907 +0000 UTC m=+18.199227958 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret") pod "global-pull-secret-syncer-rczng" (UID: "b87ad709-411d-4c4c-81a7-100bf4ac44d5") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:25:41.357583 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:41.357545 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74rv\" (UniqueName: \"kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv\") pod \"network-check-target-4w7rt\" (UID: \"c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016\") " pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:41.357763 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:41.357710 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:25:41.357763 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:41.357729 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:25:41.357763 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:41.357744 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w74rv for pod openshift-network-diagnostics/network-check-target-4w7rt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:25:41.357932 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:41.357800 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv podName:c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016 nodeName:}" failed. No retries permitted until 2026-09-09 02:25:49.357781722 +0000 UTC m=+18.300350625 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w74rv" (UniqueName: "kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv") pod "network-check-target-4w7rt" (UID: "c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:25:41.576554 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:41.576472 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:41.577014 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:41.576571 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:41.577014 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:41.576875 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:41.577014 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:41.576966 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:42.575329 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:42.575285 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:42.575514 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:42.575442 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:43.575138 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:43.575045 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:43.575138 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:43.575077 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:43.575685 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:43.575176 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:43.575685 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:43.575317 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:44.575519 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:44.575482 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:44.575937 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:44.575616 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:45.575847 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:45.575749 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:45.576253 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:45.575873 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:45.576253 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:45.575962 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:45.576253 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:45.576059 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:46.575997 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:46.575960 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:46.576396 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:46.576096 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:47.575602 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:47.575564 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:47.575796 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:47.575574 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:47.575796 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:47.575693 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:47.575796 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:47.575747 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:48.575831 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:48.575794 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:48.576276 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:48.575907 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:49.315312 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:49.315273 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:49.315510 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:49.315332 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:49.315510 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:49.315404 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:25:49.315510 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:49.315441 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:25:49.315510 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:49.315475 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs podName:06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab nodeName:}" failed. No retries permitted until 2026-09-09 02:26:05.315456704 +0000 UTC m=+34.258025612 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs") pod "network-metrics-daemon-szmkq" (UID: "06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:25:49.315510 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:49.315490 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret podName:b87ad709-411d-4c4c-81a7-100bf4ac44d5 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:05.315483613 +0000 UTC m=+34.258052504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret") pod "global-pull-secret-syncer-rczng" (UID: "b87ad709-411d-4c4c-81a7-100bf4ac44d5") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:25:49.416782 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:49.416746 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74rv\" (UniqueName: \"kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv\") pod \"network-check-target-4w7rt\" (UID: \"c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016\") " pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:49.416962 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:49.416902 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:25:49.416962 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:49.416923 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:25:49.416962 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:49.416935 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w74rv for pod openshift-network-diagnostics/network-check-target-4w7rt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:25:49.417064 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:49.416995 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv podName:c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:05.416976839 +0000 UTC m=+34.359545733 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w74rv" (UniqueName: "kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv") pod "network-check-target-4w7rt" (UID: "c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:25:49.575269 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:49.575191 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:49.575269 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:49.575236 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:49.575489 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:49.575356 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:49.575489 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:49.575446 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:50.575277 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:50.575244 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:50.575767 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:50.575381 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:51.576010 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:51.575956 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:51.576457 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:51.576068 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:51.576457 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:51.576126 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:51.576457 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:51.576227 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:52.575112 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.574950 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:52.575215 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:52.575197 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:52.687295 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.687263 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" event={"ID":"3195ff7f-9445-49b7-a438-2fcb5667f15b","Type":"ContainerStarted","Data":"702b1938c9c4fa49a49f1e5ee5ff19a248b57c1b6b47d6a9d6a74ae85d434b5d"} Sep 09 02:25:52.688758 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.688732 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" event={"ID":"70038143-027a-442d-bc5f-c8511317e7b5","Type":"ContainerStarted","Data":"62d9118481e9f2f0457f321f4c6b71bf0764cfc64abc84f1e973ddb5b9344aaf"} Sep 09 02:25:52.691579 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.691554 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" event={"ID":"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9","Type":"ContainerStarted","Data":"fe346ba997ab1c956bf9e687f4f1b7d2ebad4cd6b6fadc35acb66fb218205ac8"} Sep 09 02:25:52.691707 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.691648 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" event={"ID":"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9","Type":"ContainerStarted","Data":"d09a9eadd95f200bd578443b635ef6e789f9f813f5b5bc196c09cde2b3526f5f"} Sep 09 02:25:52.691707 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.691703 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" event={"ID":"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9","Type":"ContainerStarted","Data":"c037e80d6735f1a8f7eebd4c3909f5be764a6b81e4a60f7e92553800424e15e5"} Sep 09 02:25:52.691878 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.691753 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" event={"ID":"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9","Type":"ContainerStarted","Data":"1071c953a55d261b45448b3bf28e254ec926ad83403e8513a29da02e722056de"} Sep 09 02:25:52.691878 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.691794 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" event={"ID":"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9","Type":"ContainerStarted","Data":"b41408f72e38963a0fe4c77c0c25200c4b87b76569483cb2c5fe7cf79eaa5f0c"} Sep 09 02:25:52.693051 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.693027 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4cghs" event={"ID":"454c2cf7-2e0e-4f6a-a40e-33965988ce44","Type":"ContainerStarted","Data":"9002e99abfb1d29197a2da24d0d7e0313c3a3c9b236a0e210bf073ee319dddf5"} Sep 09 02:25:52.697439 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.697409 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-smfw9" event={"ID":"953c8a38-a75f-4767-9866-68b3336a65fb","Type":"ContainerStarted","Data":"458974a881cf92f5150e7cab51b2001f2d399f133e93dd7a02289c8280f87f19"} Sep 09 02:25:52.699029 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.699004 2577 generic.go:356] "Generic (PLEG): container finished" podID="b6b7de4b-5150-40f8-9ef4-f6c50994bb97" containerID="ad0dbeaf0823f0169457c06b1a0a93de5ffe756cfd01e00e4b129d0a8c9185b5" exitCode=0 Sep 09 02:25:52.699122 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.699033 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97k2r" event={"ID":"b6b7de4b-5150-40f8-9ef4-f6c50994bb97","Type":"ContainerDied","Data":"ad0dbeaf0823f0169457c06b1a0a93de5ffe756cfd01e00e4b129d0a8c9185b5"} Sep 09 02:25:52.700683 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.700659 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-t4fn5" event={"ID":"2f661248-5853-4ec4-99ef-f03ae698507b","Type":"ContainerStarted","Data":"1a2186c16ed193f6baa3734e244e43027c9691c09b214aa8e5381a5232be7a21"} Sep 09 02:25:52.702031 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.702014 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sffcl" event={"ID":"d5d907cf-31d4-42d9-b7e1-69ee4f71c967","Type":"ContainerStarted","Data":"9445c0a3c559adc0e5f8c328ea265a32df31ce67a7b0051be09cd08e59c2bb5d"} Sep 09 02:25:52.705992 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.705959 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-lpvvz" podStartSLOduration=3.060724032 podStartE2EDuration="20.70594879s" totalImagesPullingTime="17.645224758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:25:52.705705866 +0000 UTC m=+21.648274776" watchObservedRunningTime="2026-09-09 02:25:52.70594879 +0000 UTC m=+21.648517700" Sep 09 02:25:52.717822 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.717782 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-smfw9" podStartSLOduration=4.060892046 podStartE2EDuration="21.717766599s" totalImagesPullingTime="17.656874553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:25:52.717739902 +0000 UTC m=+21.660308815" watchObservedRunningTime="2026-09-09 02:25:52.717766599 +0000 UTC m=+21.660335509" Sep 09 02:25:52.757601 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.757567 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-t4fn5" podStartSLOduration=4.102902844 podStartE2EDuration="21.757554114s" totalImagesPullingTime="17.65465127s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:25:52.757183572 +0000 UTC m=+21.699752482" watchObservedRunningTime="2026-09-09 02:25:52.757554114 +0000 UTC m=+21.700123023" Sep 09 02:25:52.772010 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.771962 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4cghs" podStartSLOduration=4.053167321 podStartE2EDuration="21.771946513s" totalImagesPullingTime="17.718779192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:25:52.771833873 +0000 UTC m=+21.714402782" watchObservedRunningTime="2026-09-09 02:25:52.771946513 +0000 UTC m=+21.714515424" Sep 09 02:25:52.786592 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:52.786554 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sffcl" podStartSLOduration=4.116702006 podStartE2EDuration="21.786539733s" totalImagesPullingTime="17.669837727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:25:52.786110235 +0000 UTC m=+21.728679140" watchObservedRunningTime="2026-09-09 02:25:52.786539733 +0000 UTC m=+21.729108642" Sep 09 02:25:53.575975 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:53.575942 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:53.576189 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:53.575947 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:53.576189 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:53.576058 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:53.576189 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:53.576134 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:53.707944 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:53.707904 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" event={"ID":"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9","Type":"ContainerStarted","Data":"26f3a4e22669bdf52e30afb37afbd350d3b87fae829a78ff87fa9d7248a19b0e"} Sep 09 02:25:53.709535 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:53.709472 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-blm2g" event={"ID":"b24f78fe-031a-4be0-b88f-247678a0b92d","Type":"ContainerStarted","Data":"6a47059c41e4d3d9799ebe2b25275a3310677e61f9001fe240ab32c090f9dbc9"} Sep 09 02:25:53.723287 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:53.723241 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-blm2g" podStartSLOduration=5.0742394730000004 podStartE2EDuration="22.723227629s" totalImagesPullingTime="17.648988156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:25:53.722894226 +0000 UTC m=+22.665463146" watchObservedRunningTime="2026-09-09 02:25:53.723227629 +0000 UTC m=+22.665796536" Sep 09 02:25:53.914007 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:53.913976 2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 02:25:54.543235 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:54.543143 2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T02:25:53.914001325Z","UUID":"901007c2-c100-48d0-91cf-82e2eec94406","Handler":null,"Name":"","Endpoint":""} Sep 09 02:25:54.546237 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:54.546214 2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 02:25:54.546369 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:54.546247 2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 02:25:54.575010 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:54.574989 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:54.575117 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:54.575086 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:54.714238 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:54.714200 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" event={"ID":"3195ff7f-9445-49b7-a438-2fcb5667f15b","Type":"ContainerStarted","Data":"181840fa4c9f6f9ec597fa8438d96b388358e2fe1929c6bdb1dbe0cb24de7b2c"} Sep 09 02:25:55.152173 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:55.151496 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-smfw9" Sep 09 02:25:55.152360 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:55.152213 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-smfw9" Sep 09 02:25:55.575613 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:55.575516 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:55.575613 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:55.575561 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:55.575848 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:55.575694 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:55.575848 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:55.575821 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:55.719117 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:55.719076 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" event={"ID":"3195ff7f-9445-49b7-a438-2fcb5667f15b","Type":"ContainerStarted","Data":"6561e5378c9bb021ac33672b9c8e16823706c38273c2d21397af6c91c73fae48"} Sep 09 02:25:55.722817 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:55.722786 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" event={"ID":"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9","Type":"ContainerStarted","Data":"bfb41fceda165e21469cd91a2e7f72e335a58e57e96b1bef21598a4d59e51dd6"} Sep 09 02:25:55.722963 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:55.722886 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-smfw9" Sep 09 02:25:55.723455 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:55.723436 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-smfw9" Sep 09 02:25:55.736592 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:55.736551 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-rpsxx" podStartSLOduration=4.52918079 podStartE2EDuration="24.736538298s" totalImagesPullingTime="20.207357508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:25:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:25:55.736475268 +0000 UTC m=+24.679044189" watchObservedRunningTime="2026-09-09 02:25:55.736538298 +0000 UTC m=+24.679107208" Sep 09 02:25:56.575359 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:56.575324 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:56.575535 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:56.575458 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:57.575945 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:57.575912 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:57.575945 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:57.575927 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:57.576558 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:57.576025 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:57.576558 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:57.576147 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:58.576102 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:58.575905 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:58.576507 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:58.576143 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:25:58.732762 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:58.732723 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" event={"ID":"8dfc061f-fdd5-4e0d-8a05-9e3219360bd9","Type":"ContainerStarted","Data":"960bf58f2790f7a5651ca298aaa0f634a0a1499b5892400216585592f0a147c2"} Sep 09 02:25:58.732955 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:58.732938 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:58.734646 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:58.734600 2577 generic.go:356] "Generic (PLEG): container finished" podID="b6b7de4b-5150-40f8-9ef4-f6c50994bb97" containerID="1b9f91fa6be5635a8953400f9e2ff0bd3c802d892ff889cacf90bf5c9a358cf0" exitCode=0 Sep 09 02:25:58.734761 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:58.734649 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97k2r" event={"ID":"b6b7de4b-5150-40f8-9ef4-f6c50994bb97","Type":"ContainerDied","Data":"1b9f91fa6be5635a8953400f9e2ff0bd3c802d892ff889cacf90bf5c9a358cf0"} Sep 09 02:25:58.748007 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:58.747986 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:58.758500 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:58.758460 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" podStartSLOduration=10.045931383 podStartE2EDuration="27.758447556s" totalImagesPullingTime="17.712516173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:25:58.756561464 +0000 UTC m=+27.699130374" watchObservedRunningTime="2026-09-09 02:25:58.758447556 +0000 UTC m=+27.701016465" Sep 09 02:25:59.575086 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:59.575047 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:59.575253 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:59.575170 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:59.575253 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:59.575226 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:59.575376 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:59.575348 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:59.737566 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:59.737534 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 09 02:25:59.738114 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:59.738065 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:59.753861 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:59.753831 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:25:59.987481 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:59.987450 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rczng"] Sep 09 02:25:59.987607 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:59.987592 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:25:59.987724 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:59.987705 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:25:59.989662 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:59.989583 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4w7rt"] Sep 09 02:25:59.989765 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:59.989691 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:25:59.989765 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:59.989760 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:25:59.992907 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:59.992869 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-szmkq"] Sep 09 02:25:59.993008 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:25:59.992974 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:25:59.993080 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:25:59.993060 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:26:00.741466 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:00.741430 2577 generic.go:356] "Generic (PLEG): container finished" podID="b6b7de4b-5150-40f8-9ef4-f6c50994bb97" containerID="0ba80e0dbb58f018f921236043c261633b7a323e0f1350570e19f973cd486b4d" exitCode=0 Sep 09 02:26:00.741932 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:00.741514 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97k2r" event={"ID":"b6b7de4b-5150-40f8-9ef4-f6c50994bb97","Type":"ContainerDied","Data":"0ba80e0dbb58f018f921236043c261633b7a323e0f1350570e19f973cd486b4d"} Sep 09 02:26:00.741932 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:00.741576 2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 09 02:26:01.166225 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:01.166179 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:26:01.579414 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:01.579386 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:26:01.579589 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:01.579466 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:26:01.579589 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:01.579523 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:26:01.579695 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:01.579612 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:26:01.579695 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:01.579660 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:26:01.579791 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:01.579697 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:26:02.748169 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:02.747987 2577 generic.go:356] "Generic (PLEG): container finished" podID="b6b7de4b-5150-40f8-9ef4-f6c50994bb97" containerID="67573352f903629c03ca47e299183ec62abef62bd3b5d6c4ca590e649147efa8" exitCode=0 Sep 09 02:26:02.748522 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:02.748048 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97k2r" event={"ID":"b6b7de4b-5150-40f8-9ef4-f6c50994bb97","Type":"ContainerDied","Data":"67573352f903629c03ca47e299183ec62abef62bd3b5d6c4ca590e649147efa8"} Sep 09 02:26:03.575623 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:03.575590 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:26:03.575810 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:03.575652 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:26:03.575810 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:03.575590 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:26:03.575810 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:03.575749 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-szmkq" podUID="06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab" Sep 09 02:26:03.575967 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:03.575863 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rczng" podUID="b87ad709-411d-4c4c-81a7-100bf4ac44d5" Sep 09 02:26:03.575967 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:03.575952 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4w7rt" podUID="c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016" Sep 09 02:26:04.868563 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.868474 2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-153.ec2.internal" event="NodeReady" Sep 09 02:26:04.868954 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.868605 2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 02:26:04.901504 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.901471 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c69dff986-kcc28"] Sep 09 02:26:04.905254 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.905231 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:04.908087 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.908031 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 09 02:26:04.908087 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.908031 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h655c\"" Sep 09 02:26:04.908429 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.908410 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 09 02:26:04.908586 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.908572 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 09 02:26:04.914282 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.914054 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7nbg2"] Sep 09 02:26:04.914564 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.914532 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 09 02:26:04.917745 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.917722 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c69dff986-kcc28"] Sep 09 02:26:04.917834 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.917812 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:04.919448 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.919426 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-92stt"] Sep 09 02:26:04.920528 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.920446 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t86xl\"" Sep 09 02:26:04.920528 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.920449 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 02:26:04.920528 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.920479 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 02:26:04.922995 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.922976 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-92stt" Sep 09 02:26:04.925324 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.925303 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 02:26:04.925422 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.925321 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 02:26:04.925422 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.925341 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 02:26:04.925543 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.925526 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-g2z42\"" Sep 09 02:26:04.928878 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.928821 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7nbg2"] Sep 09 02:26:04.933707 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:04.933688 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-92stt"] Sep 09 02:26:05.029803 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.029765 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:05.029990 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.029820 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2d68c24-03df-44b2-9bf1-5ef24b24577b-installation-pull-secrets\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.029990 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.029848 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-bound-sa-token\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.029990 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.029873 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cz5r\" (UniqueName: \"kubernetes.io/projected/34a35235-50fe-4c00-970b-7d1d13fb59aa-kube-api-access-8cz5r\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:05.029990 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.029909 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2d68c24-03df-44b2-9bf1-5ef24b24577b-trusted-ca\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.029990 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.029976 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxqwj\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-kube-api-access-rxqwj\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.030302 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.030007 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a35235-50fe-4c00-970b-7d1d13fb59aa-tmp-dir\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:05.030302 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.030078 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2d68c24-03df-44b2-9bf1-5ef24b24577b-ca-trust-extracted\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.030302 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.030115 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f73d14e5-2b62-4cb1-962d-c2b4b74446f7-cert\") pod \"ingress-canary-92stt\" (UID: \"f73d14e5-2b62-4cb1-962d-c2b4b74446f7\") " pod="openshift-ingress-canary/ingress-canary-92stt" Sep 09 02:26:05.030302 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.030166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.030302 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.030197 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-certificates\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.030302 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.030221 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7s9p\" (UniqueName: \"kubernetes.io/projected/f73d14e5-2b62-4cb1-962d-c2b4b74446f7-kube-api-access-t7s9p\") pod \"ingress-canary-92stt\" (UID: \"f73d14e5-2b62-4cb1-962d-c2b4b74446f7\") " pod="openshift-ingress-canary/ingress-canary-92stt" Sep 09 02:26:05.030537 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.030344 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2d68c24-03df-44b2-9bf1-5ef24b24577b-image-registry-private-configuration\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.030537 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.030374 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34a35235-50fe-4c00-970b-7d1d13fb59aa-config-volume\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:05.131058 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.130978 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-certificates\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.131058 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.131017 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7s9p\" (UniqueName: \"kubernetes.io/projected/f73d14e5-2b62-4cb1-962d-c2b4b74446f7-kube-api-access-t7s9p\") pod \"ingress-canary-92stt\" (UID: \"f73d14e5-2b62-4cb1-962d-c2b4b74446f7\") " pod="openshift-ingress-canary/ingress-canary-92stt" Sep 09 02:26:05.131058 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.131045 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2d68c24-03df-44b2-9bf1-5ef24b24577b-image-registry-private-configuration\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.131334 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.131071 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34a35235-50fe-4c00-970b-7d1d13fb59aa-config-volume\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:05.131334 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.131247 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:05.131334 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.131315 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2d68c24-03df-44b2-9bf1-5ef24b24577b-installation-pull-secrets\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.131488 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.131341 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-bound-sa-token\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.131488 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.131357 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cz5r\" (UniqueName: \"kubernetes.io/projected/34a35235-50fe-4c00-970b-7d1d13fb59aa-kube-api-access-8cz5r\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:05.131488 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.131407 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 02:26:05.131879 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.131853 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls podName:34a35235-50fe-4c00-970b-7d1d13fb59aa nodeName:}" failed. No retries permitted until 2026-09-09 02:26:05.631448825 +0000 UTC m=+34.574017719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls") pod "dns-default-7nbg2" (UID: "34a35235-50fe-4c00-970b-7d1d13fb59aa") : secret "dns-default-metrics-tls" not found Sep 09 02:26:05.131987 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.131892 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2d68c24-03df-44b2-9bf1-5ef24b24577b-trusted-ca\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.131987 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.131938 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxqwj\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-kube-api-access-rxqwj\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.131987 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.131962 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a35235-50fe-4c00-970b-7d1d13fb59aa-tmp-dir\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:05.132088 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.132019 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2d68c24-03df-44b2-9bf1-5ef24b24577b-ca-trust-extracted\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.132088 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.132026 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/34a35235-50fe-4c00-970b-7d1d13fb59aa-config-volume\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:05.132088 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.132038 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-certificates\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.132088 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.132052 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f73d14e5-2b62-4cb1-962d-c2b4b74446f7-cert\") pod \"ingress-canary-92stt\" (UID: \"f73d14e5-2b62-4cb1-962d-c2b4b74446f7\") " pod="openshift-ingress-canary/ingress-canary-92stt" Sep 09 02:26:05.132247 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.132221 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 02:26:05.132247 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.132244 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c69dff986-kcc28: secret "image-registry-tls" not found Sep 09 02:26:05.132348 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.132305 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls podName:e2d68c24-03df-44b2-9bf1-5ef24b24577b nodeName:}" failed. No retries permitted until 2026-09-09 02:26:05.632288664 +0000 UTC m=+34.574857552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls") pod "image-registry-c69dff986-kcc28" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b") : secret "image-registry-tls" not found Sep 09 02:26:05.133621 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.133594 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34a35235-50fe-4c00-970b-7d1d13fb59aa-tmp-dir\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:05.134237 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.134196 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2d68c24-03df-44b2-9bf1-5ef24b24577b-trusted-ca\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.134324 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.134259 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2d68c24-03df-44b2-9bf1-5ef24b24577b-ca-trust-extracted\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.134384 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.134321 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.136402 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.136380 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f73d14e5-2b62-4cb1-962d-c2b4b74446f7-cert\") pod \"ingress-canary-92stt\" (UID: \"f73d14e5-2b62-4cb1-962d-c2b4b74446f7\") " pod="openshift-ingress-canary/ingress-canary-92stt" Sep 09 02:26:05.136501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.136379 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2d68c24-03df-44b2-9bf1-5ef24b24577b-image-registry-private-configuration\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.137594 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.137565 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2d68c24-03df-44b2-9bf1-5ef24b24577b-installation-pull-secrets\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.139973 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.139952 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7s9p\" (UniqueName: \"kubernetes.io/projected/f73d14e5-2b62-4cb1-962d-c2b4b74446f7-kube-api-access-t7s9p\") pod \"ingress-canary-92stt\" (UID: \"f73d14e5-2b62-4cb1-962d-c2b4b74446f7\") " pod="openshift-ingress-canary/ingress-canary-92stt" Sep 09 02:26:05.140876 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.140858 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-bound-sa-token\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.141238 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.141210 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxqwj\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-kube-api-access-rxqwj\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.141883 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.141859 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cz5r\" (UniqueName: \"kubernetes.io/projected/34a35235-50fe-4c00-970b-7d1d13fb59aa-kube-api-access-8cz5r\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:05.237670 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.237617 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-92stt" Sep 09 02:26:05.335244 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.335069 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:26:05.335244 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.335248 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:26:05.335421 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.335221 2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:26:05.335421 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.335358 2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 02:26:05.335421 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.335362 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs podName:06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab nodeName:}" failed. No retries permitted until 2026-09-09 02:26:37.335345757 +0000 UTC m=+66.277914650 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs") pod "network-metrics-daemon-szmkq" (UID: "06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 02:26:05.335421 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.335412 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret podName:b87ad709-411d-4c4c-81a7-100bf4ac44d5 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:37.335397143 +0000 UTC m=+66.277966031 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret") pod "global-pull-secret-syncer-rczng" (UID: "b87ad709-411d-4c4c-81a7-100bf4ac44d5") : object "kube-system"/"original-pull-secret" not registered Sep 09 02:26:05.435804 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.435712 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74rv\" (UniqueName: \"kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv\") pod \"network-check-target-4w7rt\" (UID: \"c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016\") " pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:26:05.435965 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.435899 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 02:26:05.435965 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.435923 2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 02:26:05.435965 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.435935 2577 projected.go:196] Error preparing data for projected volume kube-api-access-w74rv for pod openshift-network-diagnostics/network-check-target-4w7rt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:26:05.436119 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.436002 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv podName:c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:37.435982776 +0000 UTC m=+66.378551684 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w74rv" (UniqueName: "kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv") pod "network-check-target-4w7rt" (UID: "c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 02:26:05.521716 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.521662 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-92stt"] Sep 09 02:26:05.526174 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:05.526147 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73d14e5_2b62_4cb1_962d_c2b4b74446f7.slice/crio-b47feceb5af5b9a43f95a84706ffd8760804259ed969b58ebb157258d537f113 WatchSource:0}: Error finding container b47feceb5af5b9a43f95a84706ffd8760804259ed969b58ebb157258d537f113: Status 404 returned error can't find the container with id b47feceb5af5b9a43f95a84706ffd8760804259ed969b58ebb157258d537f113 Sep 09 02:26:05.570539 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.570507 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-225b5"] Sep 09 02:26:05.575349 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.575327 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-225b5" Sep 09 02:26:05.578151 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.578121 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 09 02:26:05.578273 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.578150 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 09 02:26:05.578747 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.578729 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:26:05.578886 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.578870 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:26:05.579040 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.579014 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-k9hgx\"" Sep 09 02:26:05.579124 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.579056 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:26:05.581057 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.581043 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 02:26:05.581467 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.581448 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 02:26:05.581708 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.581691 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 02:26:05.581708 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.581703 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 02:26:05.581934 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.581917 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-swb8h\"" Sep 09 02:26:05.582141 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.582127 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v2n6b\"" Sep 09 02:26:05.583174 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.583052 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-225b5"] Sep 09 02:26:05.587026 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.586977 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-d4vlb"] Sep 09 02:26:05.590926 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.590909 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc"] Sep 09 02:26:05.591022 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.590990 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.594119 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.594098 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 09 02:26:05.594219 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.594115 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 09 02:26:05.594317 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.594294 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dj2gh\"" Sep 09 02:26:05.594531 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.594509 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" Sep 09 02:26:05.594676 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.594518 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 09 02:26:05.594868 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.594849 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 09 02:26:05.597898 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.597747 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 09 02:26:05.597898 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.597760 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-5kqf9\"" Sep 09 02:26:05.597898 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.597894 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 09 02:26:05.598101 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.597994 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 09 02:26:05.598101 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.598096 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 09 02:26:05.608899 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.608877 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 09 02:26:05.609499 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.609479 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc"] Sep 09 02:26:05.610373 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.610355 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-d4vlb"] Sep 09 02:26:05.637193 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.637166 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:05.637365 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.637264 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:05.637365 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.637312 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 02:26:05.637488 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.637384 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 02:26:05.637488 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.637397 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls podName:34a35235-50fe-4c00-970b-7d1d13fb59aa nodeName:}" failed. No retries permitted until 2026-09-09 02:26:06.637375181 +0000 UTC m=+35.579944084 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls") pod "dns-default-7nbg2" (UID: "34a35235-50fe-4c00-970b-7d1d13fb59aa") : secret "dns-default-metrics-tls" not found Sep 09 02:26:05.637488 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.637399 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c69dff986-kcc28: secret "image-registry-tls" not found Sep 09 02:26:05.637488 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.637437 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls podName:e2d68c24-03df-44b2-9bf1-5ef24b24577b nodeName:}" failed. No retries permitted until 2026-09-09 02:26:06.637427689 +0000 UTC m=+35.579996586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls") pod "image-registry-c69dff986-kcc28" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b") : secret "image-registry-tls" not found Sep 09 02:26:05.672667 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.672621 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb"] Sep 09 02:26:05.677223 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.677201 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" Sep 09 02:26:05.679602 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.679565 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h"] Sep 09 02:26:05.679928 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.679819 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 09 02:26:05.679928 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.679816 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 09 02:26:05.680704 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.680684 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 09 02:26:05.680911 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.680886 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-cz25d\"" Sep 09 02:26:05.680965 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.680958 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 09 02:26:05.687557 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.687508 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-lh5dp"] Sep 09 02:26:05.687744 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.687723 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:05.690341 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.690322 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 09 02:26:05.690472 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.690456 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 09 02:26:05.690531 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.690392 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 09 02:26:05.690762 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.690741 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4r6k2\"" Sep 09 02:26:05.690824 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.690795 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6b49ccfd44-hkkz2"] Sep 09 02:26:05.691038 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.691024 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.693739 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.693243 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 09 02:26:05.693982 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.693964 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 02:26:05.694223 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.694193 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 02:26:05.694430 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.694415 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 09 02:26:05.694723 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.694708 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-l54ks\"" Sep 09 02:26:05.695219 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.695198 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-l47l7"] Sep 09 02:26:05.695393 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.695373 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.700279 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.700259 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 09 02:26:05.700443 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.700427 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 09 02:26:05.700552 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.700532 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 09 02:26:05.700675 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.700621 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 09 02:26:05.700675 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.700460 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 09 02:26:05.700793 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.700707 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 09 02:26:05.700937 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.700923 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 09 02:26:05.701273 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.701060 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb"] Sep 09 02:26:05.701273 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.701087 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-l47l7"] Sep 09 02:26:05.701273 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.701107 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h"] Sep 09 02:26:05.701273 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.701132 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6b49ccfd44-hkkz2"] Sep 09 02:26:05.701273 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.701169 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-l47l7" Sep 09 02:26:05.702879 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.702859 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-dks4w\"" Sep 09 02:26:05.703842 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.703825 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2rg86\"" Sep 09 02:26:05.705832 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.705814 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-lh5dp"] Sep 09 02:26:05.738428 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.738405 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96q4s\" (UniqueName: \"kubernetes.io/projected/6ff5de1e-f270-439b-a4e0-3bb1394ecb79-kube-api-access-96q4s\") pod \"volume-data-source-validator-695c895bd6-225b5\" (UID: \"6ff5de1e-f270-439b-a4e0-3bb1394ecb79\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-225b5" Sep 09 02:26:05.738559 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.738449 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh87q\" (UniqueName: \"kubernetes.io/projected/58860e10-d2bc-4a1a-8b98-20d772421b9f-kube-api-access-mh87q\") pod \"console-operator-85b454874b-d4vlb\" (UID: \"58860e10-d2bc-4a1a-8b98-20d772421b9f\") " pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.738559 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.738468 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3018c2e4-70dc-4421-8bb0-540126c40bd5-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-ttjnc\" (UID: \"3018c2e4-70dc-4421-8bb0-540126c40bd5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" Sep 09 02:26:05.738662 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.738566 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58860e10-d2bc-4a1a-8b98-20d772421b9f-config\") pod \"console-operator-85b454874b-d4vlb\" (UID: \"58860e10-d2bc-4a1a-8b98-20d772421b9f\") " pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.738662 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.738593 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58860e10-d2bc-4a1a-8b98-20d772421b9f-trusted-ca\") pod \"console-operator-85b454874b-d4vlb\" (UID: \"58860e10-d2bc-4a1a-8b98-20d772421b9f\") " pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.738662 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.738646 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3018c2e4-70dc-4421-8bb0-540126c40bd5-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-ttjnc\" (UID: \"3018c2e4-70dc-4421-8bb0-540126c40bd5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" Sep 09 02:26:05.738831 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.738674 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmrq2\" (UniqueName: \"kubernetes.io/projected/3018c2e4-70dc-4421-8bb0-540126c40bd5-kube-api-access-mmrq2\") pod \"kube-storage-version-migrator-operator-8d7dd4675-ttjnc\" (UID: \"3018c2e4-70dc-4421-8bb0-540126c40bd5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" Sep 09 02:26:05.738831 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.738710 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58860e10-d2bc-4a1a-8b98-20d772421b9f-serving-cert\") pod \"console-operator-85b454874b-d4vlb\" (UID: \"58860e10-d2bc-4a1a-8b98-20d772421b9f\") " pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.757949 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.757912 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-92stt" event={"ID":"f73d14e5-2b62-4cb1-962d-c2b4b74446f7","Type":"ContainerStarted","Data":"b47feceb5af5b9a43f95a84706ffd8760804259ed969b58ebb157258d537f113"} Sep 09 02:26:05.772843 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.772815 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5"] Sep 09 02:26:05.776368 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.776349 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:05.779030 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.779008 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 02:26:05.779173 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.779101 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 02:26:05.779372 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.779352 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2nfst\"" Sep 09 02:26:05.780529 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.780513 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 09 02:26:05.780780 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.780514 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 09 02:26:05.788136 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.788117 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5"] Sep 09 02:26:05.837778 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.837745 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:05.839813 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.839790 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3018c2e4-70dc-4421-8bb0-540126c40bd5-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-ttjnc\" (UID: \"3018c2e4-70dc-4421-8bb0-540126c40bd5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" Sep 09 02:26:05.839936 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.839830 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmrq2\" (UniqueName: \"kubernetes.io/projected/3018c2e4-70dc-4421-8bb0-540126c40bd5-kube-api-access-mmrq2\") pod \"kube-storage-version-migrator-operator-8d7dd4675-ttjnc\" (UID: \"3018c2e4-70dc-4421-8bb0-540126c40bd5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" Sep 09 02:26:05.839936 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.839859 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1183493c-bda9-4396-858e-ee95549d8a22-serving-cert\") pod \"service-ca-operator-859f94c4b-k87wb\" (UID: \"1183493c-bda9-4396-858e-ee95549d8a22\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" Sep 09 02:26:05.839936 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.839912 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58860e10-d2bc-4a1a-8b98-20d772421b9f-config\") pod \"console-operator-85b454874b-d4vlb\" (UID: \"58860e10-d2bc-4a1a-8b98-20d772421b9f\") " pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.840197 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840173 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtkhb\" (UniqueName: \"kubernetes.io/projected/1afc5697-33db-4342-8de4-e1145ad105ef-kube-api-access-wtkhb\") pod \"cluster-samples-operator-7cdcfcf76c-7hs9h\" (UID: \"1afc5697-33db-4342-8de4-e1145ad105ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:05.840265 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840249 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1183493c-bda9-4396-858e-ee95549d8a22-config\") pod \"service-ca-operator-859f94c4b-k87wb\" (UID: \"1183493c-bda9-4396-858e-ee95549d8a22\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" Sep 09 02:26:05.840329 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840277 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcljq\" (UniqueName: \"kubernetes.io/projected/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-kube-api-access-jcljq\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.840329 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840303 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv79t\" (UniqueName: \"kubernetes.io/projected/28b8bb88-2eef-4b7a-bd55-16549de8228f-kube-api-access-zv79t\") pod \"network-check-source-685456655f-l47l7\" (UID: \"28b8bb88-2eef-4b7a-bd55-16549de8228f\") " pod="openshift-network-diagnostics/network-check-source-685456655f-l47l7" Sep 09 02:26:05.840329 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840321 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15536224-40d3-43d7-b5df-c2a44121f29d-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.840503 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840359 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58860e10-d2bc-4a1a-8b98-20d772421b9f-serving-cert\") pod \"console-operator-85b454874b-d4vlb\" (UID: \"58860e10-d2bc-4a1a-8b98-20d772421b9f\") " pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.840503 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840384 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grkfp\" (UniqueName: \"kubernetes.io/projected/15536224-40d3-43d7-b5df-c2a44121f29d-kube-api-access-grkfp\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.840503 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840412 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.840503 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840449 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15536224-40d3-43d7-b5df-c2a44121f29d-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.840503 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840477 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15536224-40d3-43d7-b5df-c2a44121f29d-serving-cert\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.840717 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840503 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96q4s\" (UniqueName: \"kubernetes.io/projected/6ff5de1e-f270-439b-a4e0-3bb1394ecb79-kube-api-access-96q4s\") pod \"volume-data-source-validator-695c895bd6-225b5\" (UID: \"6ff5de1e-f270-439b-a4e0-3bb1394ecb79\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-225b5" Sep 09 02:26:05.840717 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840525 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-default-certificate\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.840717 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840549 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-stats-auth\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.840717 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840571 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15536224-40d3-43d7-b5df-c2a44121f29d-tmp\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.840717 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840590 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58860e10-d2bc-4a1a-8b98-20d772421b9f-config\") pod \"console-operator-85b454874b-d4vlb\" (UID: \"58860e10-d2bc-4a1a-8b98-20d772421b9f\") " pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.840717 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840595 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3018c2e4-70dc-4421-8bb0-540126c40bd5-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-ttjnc\" (UID: \"3018c2e4-70dc-4421-8bb0-540126c40bd5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" Sep 09 02:26:05.840717 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840672 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58860e10-d2bc-4a1a-8b98-20d772421b9f-trusted-ca\") pod \"console-operator-85b454874b-d4vlb\" (UID: \"58860e10-d2bc-4a1a-8b98-20d772421b9f\") " pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.841241 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.840702 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.841490 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.841467 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7hs9h\" (UID: \"1afc5697-33db-4342-8de4-e1145ad105ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:05.841490 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.841482 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3018c2e4-70dc-4421-8bb0-540126c40bd5-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-ttjnc\" (UID: \"3018c2e4-70dc-4421-8bb0-540126c40bd5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" Sep 09 02:26:05.841878 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.841851 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/15536224-40d3-43d7-b5df-c2a44121f29d-snapshots\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.842085 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.842058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9f59\" (UniqueName: \"kubernetes.io/projected/1183493c-bda9-4396-858e-ee95549d8a22-kube-api-access-x9f59\") pod \"service-ca-operator-859f94c4b-k87wb\" (UID: \"1183493c-bda9-4396-858e-ee95549d8a22\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" Sep 09 02:26:05.842187 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.842119 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh87q\" (UniqueName: \"kubernetes.io/projected/58860e10-d2bc-4a1a-8b98-20d772421b9f-kube-api-access-mh87q\") pod \"console-operator-85b454874b-d4vlb\" (UID: \"58860e10-d2bc-4a1a-8b98-20d772421b9f\") " pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.845376 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.843581 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/58860e10-d2bc-4a1a-8b98-20d772421b9f-trusted-ca\") pod \"console-operator-85b454874b-d4vlb\" (UID: \"58860e10-d2bc-4a1a-8b98-20d772421b9f\") " pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.845889 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.845863 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3018c2e4-70dc-4421-8bb0-540126c40bd5-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-ttjnc\" (UID: \"3018c2e4-70dc-4421-8bb0-540126c40bd5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" Sep 09 02:26:05.845982 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.845907 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58860e10-d2bc-4a1a-8b98-20d772421b9f-serving-cert\") pod \"console-operator-85b454874b-d4vlb\" (UID: \"58860e10-d2bc-4a1a-8b98-20d772421b9f\") " pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.848229 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.848200 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmrq2\" (UniqueName: \"kubernetes.io/projected/3018c2e4-70dc-4421-8bb0-540126c40bd5-kube-api-access-mmrq2\") pod \"kube-storage-version-migrator-operator-8d7dd4675-ttjnc\" (UID: \"3018c2e4-70dc-4421-8bb0-540126c40bd5\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" Sep 09 02:26:05.849354 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.849332 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96q4s\" (UniqueName: \"kubernetes.io/projected/6ff5de1e-f270-439b-a4e0-3bb1394ecb79-kube-api-access-96q4s\") pod \"volume-data-source-validator-695c895bd6-225b5\" (UID: \"6ff5de1e-f270-439b-a4e0-3bb1394ecb79\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-225b5" Sep 09 02:26:05.849437 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.849401 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh87q\" (UniqueName: \"kubernetes.io/projected/58860e10-d2bc-4a1a-8b98-20d772421b9f-kube-api-access-mh87q\") pod \"console-operator-85b454874b-d4vlb\" (UID: \"58860e10-d2bc-4a1a-8b98-20d772421b9f\") " pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.886359 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.886332 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-225b5" Sep 09 02:26:05.908993 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.908970 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" Sep 09 02:26:05.919125 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.919098 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:05.928262 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.928228 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:05.943270 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15536224-40d3-43d7-b5df-c2a44121f29d-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.943270 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943250 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x87bm\" (UniqueName: \"kubernetes.io/projected/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-kube-api-access-x87bm\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:05.943408 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943285 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15536224-40d3-43d7-b5df-c2a44121f29d-serving-cert\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.943408 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943318 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-default-certificate\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.943496 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943479 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-stats-auth\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.943548 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943512 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15536224-40d3-43d7-b5df-c2a44121f29d-tmp\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.943601 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943563 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.943684 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943595 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7hs9h\" (UID: \"1afc5697-33db-4342-8de4-e1145ad105ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:05.943684 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943654 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/15536224-40d3-43d7-b5df-c2a44121f29d-snapshots\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.943917 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943705 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9f59\" (UniqueName: \"kubernetes.io/projected/1183493c-bda9-4396-858e-ee95549d8a22-kube-api-access-x9f59\") pod \"service-ca-operator-859f94c4b-k87wb\" (UID: \"1183493c-bda9-4396-858e-ee95549d8a22\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" Sep 09 02:26:05.943917 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943738 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:05.943917 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943778 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1183493c-bda9-4396-858e-ee95549d8a22-serving-cert\") pod \"service-ca-operator-859f94c4b-k87wb\" (UID: \"1183493c-bda9-4396-858e-ee95549d8a22\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" Sep 09 02:26:05.943917 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943804 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtkhb\" (UniqueName: \"kubernetes.io/projected/1afc5697-33db-4342-8de4-e1145ad105ef-kube-api-access-wtkhb\") pod \"cluster-samples-operator-7cdcfcf76c-7hs9h\" (UID: \"1afc5697-33db-4342-8de4-e1145ad105ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:05.943917 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943834 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1183493c-bda9-4396-858e-ee95549d8a22-config\") pod \"service-ca-operator-859f94c4b-k87wb\" (UID: \"1183493c-bda9-4396-858e-ee95549d8a22\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" Sep 09 02:26:05.943917 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.943836 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 02:26:05.943917 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943860 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcljq\" (UniqueName: \"kubernetes.io/projected/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-kube-api-access-jcljq\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.943917 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943890 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv79t\" (UniqueName: \"kubernetes.io/projected/28b8bb88-2eef-4b7a-bd55-16549de8228f-kube-api-access-zv79t\") pod \"network-check-source-685456655f-l47l7\" (UID: \"28b8bb88-2eef-4b7a-bd55-16549de8228f\") " pod="openshift-network-diagnostics/network-check-source-685456655f-l47l7" Sep 09 02:26:05.943917 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.943915 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls podName:1afc5697-33db-4342-8de4-e1145ad105ef nodeName:}" failed. No retries permitted until 2026-09-09 02:26:06.443893944 +0000 UTC m=+35.386462831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-7hs9h" (UID: "1afc5697-33db-4342-8de4-e1145ad105ef") : secret "samples-operator-tls" not found Sep 09 02:26:05.944349 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943957 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:05.944349 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.943988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15536224-40d3-43d7-b5df-c2a44121f29d-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.944349 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.944035 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grkfp\" (UniqueName: \"kubernetes.io/projected/15536224-40d3-43d7-b5df-c2a44121f29d-kube-api-access-grkfp\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.944349 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.944059 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.944349 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.944130 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15536224-40d3-43d7-b5df-c2a44121f29d-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.944349 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.944213 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 02:26:05.944349 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.944239 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle podName:4f03a4a6-921a-4b9c-9bba-5de2850a8b70 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:06.44422201 +0000 UTC m=+35.386790917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle") pod "router-default-6b49ccfd44-hkkz2" (UID: "4f03a4a6-921a-4b9c-9bba-5de2850a8b70") : configmap references non-existent config key: service-ca.crt Sep 09 02:26:05.944349 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:05.944259 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs podName:4f03a4a6-921a-4b9c-9bba-5de2850a8b70 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:06.444249307 +0000 UTC m=+35.386818195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs") pod "router-default-6b49ccfd44-hkkz2" (UID: "4f03a4a6-921a-4b9c-9bba-5de2850a8b70") : secret "router-metrics-certs-default" not found Sep 09 02:26:05.944764 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.944520 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15536224-40d3-43d7-b5df-c2a44121f29d-tmp\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.944860 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.944840 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/15536224-40d3-43d7-b5df-c2a44121f29d-snapshots\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.945198 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.945174 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1183493c-bda9-4396-858e-ee95549d8a22-config\") pod \"service-ca-operator-859f94c4b-k87wb\" (UID: \"1183493c-bda9-4396-858e-ee95549d8a22\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" Sep 09 02:26:05.945285 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.945217 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15536224-40d3-43d7-b5df-c2a44121f29d-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.946469 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.946430 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-default-certificate\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.946469 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.946449 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-stats-auth\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.946757 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.946728 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15536224-40d3-43d7-b5df-c2a44121f29d-serving-cert\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.947081 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.947058 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1183493c-bda9-4396-858e-ee95549d8a22-serving-cert\") pod \"service-ca-operator-859f94c4b-k87wb\" (UID: \"1183493c-bda9-4396-858e-ee95549d8a22\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" Sep 09 02:26:05.953243 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.953223 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grkfp\" (UniqueName: \"kubernetes.io/projected/15536224-40d3-43d7-b5df-c2a44121f29d-kube-api-access-grkfp\") pod \"insights-operator-76bcd6ccf5-lh5dp\" (UID: \"15536224-40d3-43d7-b5df-c2a44121f29d\") " pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:05.953446 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.953425 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv79t\" (UniqueName: \"kubernetes.io/projected/28b8bb88-2eef-4b7a-bd55-16549de8228f-kube-api-access-zv79t\") pod \"network-check-source-685456655f-l47l7\" (UID: \"28b8bb88-2eef-4b7a-bd55-16549de8228f\") " pod="openshift-network-diagnostics/network-check-source-685456655f-l47l7" Sep 09 02:26:05.953945 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.953924 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9f59\" (UniqueName: \"kubernetes.io/projected/1183493c-bda9-4396-858e-ee95549d8a22-kube-api-access-x9f59\") pod \"service-ca-operator-859f94c4b-k87wb\" (UID: \"1183493c-bda9-4396-858e-ee95549d8a22\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" Sep 09 02:26:05.954138 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.954109 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtkhb\" (UniqueName: \"kubernetes.io/projected/1afc5697-33db-4342-8de4-e1145ad105ef-kube-api-access-wtkhb\") pod \"cluster-samples-operator-7cdcfcf76c-7hs9h\" (UID: \"1afc5697-33db-4342-8de4-e1145ad105ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:05.955817 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.955798 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcljq\" (UniqueName: \"kubernetes.io/projected/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-kube-api-access-jcljq\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:05.991903 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:05.991869 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" Sep 09 02:26:06.014333 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.014307 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" Sep 09 02:26:06.026340 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.026320 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-l47l7" Sep 09 02:26:06.045340 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.045309 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:06.045451 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.045409 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x87bm\" (UniqueName: \"kubernetes.io/projected/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-kube-api-access-x87bm\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:06.045544 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.045525 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:06.045678 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.045623 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 02:26:06.045790 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.045743 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls podName:5f61825a-519f-42c0-8ab0-3e5fcbd37b19 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:06.545719928 +0000 UTC m=+35.488288816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-xmrb5" (UID: "5f61825a-519f-42c0-8ab0-3e5fcbd37b19") : secret "cluster-monitoring-operator-tls" not found Sep 09 02:26:06.046080 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.046063 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:06.055844 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.055825 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x87bm\" (UniqueName: \"kubernetes.io/projected/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-kube-api-access-x87bm\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:06.219985 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.219918 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:06.323947 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.323917 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:06.450057 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.450019 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7hs9h\" (UID: \"1afc5697-33db-4342-8de4-e1145ad105ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:06.450223 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.450150 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 02:26:06.450270 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.450223 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls podName:1afc5697-33db-4342-8de4-e1145ad105ef nodeName:}" failed. No retries permitted until 2026-09-09 02:26:07.450205051 +0000 UTC m=+36.392773962 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-7hs9h" (UID: "1afc5697-33db-4342-8de4-e1145ad105ef") : secret "samples-operator-tls" not found Sep 09 02:26:06.450270 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.450261 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle podName:4f03a4a6-921a-4b9c-9bba-5de2850a8b70 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:07.450243702 +0000 UTC m=+36.392812613 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle") pod "router-default-6b49ccfd44-hkkz2" (UID: "4f03a4a6-921a-4b9c-9bba-5de2850a8b70") : configmap references non-existent config key: service-ca.crt Sep 09 02:26:06.450395 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.450150 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:06.450395 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.450352 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:06.450492 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.450450 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 02:26:06.450492 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.450490 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs podName:4f03a4a6-921a-4b9c-9bba-5de2850a8b70 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:07.45047738 +0000 UTC m=+36.393046300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs") pod "router-default-6b49ccfd44-hkkz2" (UID: "4f03a4a6-921a-4b9c-9bba-5de2850a8b70") : secret "router-metrics-certs-default" not found Sep 09 02:26:06.551414 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.551330 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:06.551572 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.551482 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 02:26:06.551572 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.551543 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls podName:5f61825a-519f-42c0-8ab0-3e5fcbd37b19 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:07.551525402 +0000 UTC m=+36.494094298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-xmrb5" (UID: "5f61825a-519f-42c0-8ab0-3e5fcbd37b19") : secret "cluster-monitoring-operator-tls" not found Sep 09 02:26:06.652475 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.652431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:06.652677 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.652572 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 02:26:06.652677 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.652589 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c69dff986-kcc28: secret "image-registry-tls" not found Sep 09 02:26:06.652677 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:06.652585 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:06.652677 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.652664 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls podName:e2d68c24-03df-44b2-9bf1-5ef24b24577b nodeName:}" failed. No retries permitted until 2026-09-09 02:26:08.652643995 +0000 UTC m=+37.595212897 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls") pod "image-registry-c69dff986-kcc28" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b") : secret "image-registry-tls" not found Sep 09 02:26:06.652875 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.652664 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 02:26:06.652875 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:06.652746 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls podName:34a35235-50fe-4c00-970b-7d1d13fb59aa nodeName:}" failed. No retries permitted until 2026-09-09 02:26:08.652724041 +0000 UTC m=+37.595292944 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls") pod "dns-default-7nbg2" (UID: "34a35235-50fe-4c00-970b-7d1d13fb59aa") : secret "dns-default-metrics-tls" not found Sep 09 02:26:07.413432 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.413343 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-lh5dp"] Sep 09 02:26:07.416909 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.416882 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc"] Sep 09 02:26:07.418072 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.417910 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb"] Sep 09 02:26:07.425367 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.425331 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-d4vlb"] Sep 09 02:26:07.430787 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.430765 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-225b5"] Sep 09 02:26:07.441983 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.441850 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-l47l7"] Sep 09 02:26:07.461022 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.460815 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7hs9h\" (UID: \"1afc5697-33db-4342-8de4-e1145ad105ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:07.461146 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.461132 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:07.461227 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.461213 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:07.461354 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:07.461339 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 02:26:07.461428 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:07.461418 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs podName:4f03a4a6-921a-4b9c-9bba-5de2850a8b70 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:09.461397595 +0000 UTC m=+38.403966489 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs") pod "router-default-6b49ccfd44-hkkz2" (UID: "4f03a4a6-921a-4b9c-9bba-5de2850a8b70") : secret "router-metrics-certs-default" not found Sep 09 02:26:07.461807 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:07.461769 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle podName:4f03a4a6-921a-4b9c-9bba-5de2850a8b70 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:09.461751574 +0000 UTC m=+38.404320475 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle") pod "router-default-6b49ccfd44-hkkz2" (UID: "4f03a4a6-921a-4b9c-9bba-5de2850a8b70") : configmap references non-existent config key: service-ca.crt Sep 09 02:26:07.461922 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:07.461841 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 02:26:07.461922 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:07.461891 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls podName:1afc5697-33db-4342-8de4-e1145ad105ef nodeName:}" failed. No retries permitted until 2026-09-09 02:26:09.461877741 +0000 UTC m=+38.404446649 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-7hs9h" (UID: "1afc5697-33db-4342-8de4-e1145ad105ef") : secret "samples-operator-tls" not found Sep 09 02:26:07.491716 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:07.491686 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ff5de1e_f270_439b_a4e0_3bb1394ecb79.slice/crio-8caf67a4a12b8c2e19aaa9363f34d98999f94dc1edf470b84b2c56097505c64f WatchSource:0}: Error finding container 8caf67a4a12b8c2e19aaa9363f34d98999f94dc1edf470b84b2c56097505c64f: Status 404 returned error can't find the container with id 8caf67a4a12b8c2e19aaa9363f34d98999f94dc1edf470b84b2c56097505c64f Sep 09 02:26:07.562601 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.562564 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:07.562766 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:07.562732 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 02:26:07.562819 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:07.562808 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls podName:5f61825a-519f-42c0-8ab0-3e5fcbd37b19 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:09.562787658 +0000 UTC m=+38.505356548 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-xmrb5" (UID: "5f61825a-519f-42c0-8ab0-3e5fcbd37b19") : secret "cluster-monitoring-operator-tls" not found Sep 09 02:26:07.764079 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.763999 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-225b5" event={"ID":"6ff5de1e-f270-439b-a4e0-3bb1394ecb79","Type":"ContainerStarted","Data":"8caf67a4a12b8c2e19aaa9363f34d98999f94dc1edf470b84b2c56097505c64f"} Sep 09 02:26:07.765395 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.765350 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" event={"ID":"58860e10-d2bc-4a1a-8b98-20d772421b9f","Type":"ContainerStarted","Data":"510cb92f2e76a0238e17deb9895f054f1f7fe01083ac07d999b50a926b9011a1"} Sep 09 02:26:07.768576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.768556 2577 generic.go:356] "Generic (PLEG): container finished" podID="b6b7de4b-5150-40f8-9ef4-f6c50994bb97" containerID="85ad2c48ad41951b699ac0c9c1dc2a430c600587733bbad7e6f9268b5de40172" exitCode=0 Sep 09 02:26:07.768711 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.768623 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97k2r" event={"ID":"b6b7de4b-5150-40f8-9ef4-f6c50994bb97","Type":"ContainerDied","Data":"85ad2c48ad41951b699ac0c9c1dc2a430c600587733bbad7e6f9268b5de40172"} Sep 09 02:26:07.769980 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.769954 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-l47l7" event={"ID":"28b8bb88-2eef-4b7a-bd55-16549de8228f","Type":"ContainerStarted","Data":"e5137fbffaddbe3fa30b6701219821db68a425ce14e9b31e2cb621d2cf081337"} Sep 09 02:26:07.771252 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.771222 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" event={"ID":"3018c2e4-70dc-4421-8bb0-540126c40bd5","Type":"ContainerStarted","Data":"269321cc728f327f9a88a37412f6fa24443b7324e67655da848543122e2a5c20"} Sep 09 02:26:07.772688 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.772574 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" event={"ID":"15536224-40d3-43d7-b5df-c2a44121f29d","Type":"ContainerStarted","Data":"c332c9ed8d99702a8a406249943f9be7bd9230457b831de6c05d74f89dade8aa"} Sep 09 02:26:07.773982 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:07.773962 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" event={"ID":"1183493c-bda9-4396-858e-ee95549d8a22","Type":"ContainerStarted","Data":"c6ae6a3bbf76092cf509c52d9c3894263b4110a5a72ae6f5c9c6434725c97f11"} Sep 09 02:26:08.674200 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:08.674157 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:08.674923 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:08.674297 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:08.674923 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:08.674482 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 02:26:08.674923 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:08.674541 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls podName:34a35235-50fe-4c00-970b-7d1d13fb59aa nodeName:}" failed. No retries permitted until 2026-09-09 02:26:12.674524222 +0000 UTC m=+41.617093114 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls") pod "dns-default-7nbg2" (UID: "34a35235-50fe-4c00-970b-7d1d13fb59aa") : secret "dns-default-metrics-tls" not found Sep 09 02:26:08.675114 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:08.674966 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 02:26:08.675114 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:08.674981 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c69dff986-kcc28: secret "image-registry-tls" not found Sep 09 02:26:08.675114 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:08.675025 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls podName:e2d68c24-03df-44b2-9bf1-5ef24b24577b nodeName:}" failed. No retries permitted until 2026-09-09 02:26:12.675010146 +0000 UTC m=+41.617579039 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls") pod "image-registry-c69dff986-kcc28" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b") : secret "image-registry-tls" not found Sep 09 02:26:08.786675 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:08.785728 2577 generic.go:356] "Generic (PLEG): container finished" podID="b6b7de4b-5150-40f8-9ef4-f6c50994bb97" containerID="edb3c8ac7ea5bc0d73a0d67db0d92fb9ffbf07e2b0e351a9d39d1871e7ce9a9d" exitCode=0 Sep 09 02:26:08.786675 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:08.785775 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97k2r" event={"ID":"b6b7de4b-5150-40f8-9ef4-f6c50994bb97","Type":"ContainerDied","Data":"edb3c8ac7ea5bc0d73a0d67db0d92fb9ffbf07e2b0e351a9d39d1871e7ce9a9d"} Sep 09 02:26:09.483538 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:09.482557 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:09.483538 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:09.482625 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7hs9h\" (UID: \"1afc5697-33db-4342-8de4-e1145ad105ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:09.483538 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:09.482756 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:09.483538 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:09.482920 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle podName:4f03a4a6-921a-4b9c-9bba-5de2850a8b70 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:13.482900864 +0000 UTC m=+42.425469759 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle") pod "router-default-6b49ccfd44-hkkz2" (UID: "4f03a4a6-921a-4b9c-9bba-5de2850a8b70") : configmap references non-existent config key: service-ca.crt Sep 09 02:26:09.483538 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:09.483348 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 02:26:09.483538 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:09.483406 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs podName:4f03a4a6-921a-4b9c-9bba-5de2850a8b70 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:13.483389899 +0000 UTC m=+42.425958791 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs") pod "router-default-6b49ccfd44-hkkz2" (UID: "4f03a4a6-921a-4b9c-9bba-5de2850a8b70") : secret "router-metrics-certs-default" not found Sep 09 02:26:09.483538 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:09.483466 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 02:26:09.483538 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:09.483499 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls podName:1afc5697-33db-4342-8de4-e1145ad105ef nodeName:}" failed. No retries permitted until 2026-09-09 02:26:13.483489015 +0000 UTC m=+42.426057905 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-7hs9h" (UID: "1afc5697-33db-4342-8de4-e1145ad105ef") : secret "samples-operator-tls" not found Sep 09 02:26:09.583307 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:09.583233 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:09.583468 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:09.583343 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 02:26:09.583468 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:09.583387 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls podName:5f61825a-519f-42c0-8ab0-3e5fcbd37b19 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:13.583374176 +0000 UTC m=+42.525943064 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-xmrb5" (UID: "5f61825a-519f-42c0-8ab0-3e5fcbd37b19") : secret "cluster-monitoring-operator-tls" not found Sep 09 02:26:09.606342 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:09.606302 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:09.654408 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:09.654370 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:12.710211 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:12.710166 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:12.710567 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:12.710269 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:12.710567 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:12.710327 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 02:26:12.710567 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:12.710350 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c69dff986-kcc28: secret "image-registry-tls" not found Sep 09 02:26:12.710567 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:12.710370 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 02:26:12.710567 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:12.710418 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls podName:e2d68c24-03df-44b2-9bf1-5ef24b24577b nodeName:}" failed. No retries permitted until 2026-09-09 02:26:20.710402256 +0000 UTC m=+49.652971158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls") pod "image-registry-c69dff986-kcc28" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b") : secret "image-registry-tls" not found Sep 09 02:26:12.710567 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:12.710433 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls podName:34a35235-50fe-4c00-970b-7d1d13fb59aa nodeName:}" failed. No retries permitted until 2026-09-09 02:26:20.7104274 +0000 UTC m=+49.652996288 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls") pod "dns-default-7nbg2" (UID: "34a35235-50fe-4c00-970b-7d1d13fb59aa") : secret "dns-default-metrics-tls" not found Sep 09 02:26:13.515496 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:13.515460 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:13.515496 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:13.515504 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7hs9h\" (UID: \"1afc5697-33db-4342-8de4-e1145ad105ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:13.515707 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:13.515609 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 02:26:13.515707 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:13.515609 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 02:26:13.515707 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:13.515684 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls podName:1afc5697-33db-4342-8de4-e1145ad105ef nodeName:}" failed. No retries permitted until 2026-09-09 02:26:21.515666351 +0000 UTC m=+50.458235261 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-7hs9h" (UID: "1afc5697-33db-4342-8de4-e1145ad105ef") : secret "samples-operator-tls" not found Sep 09 02:26:13.515707 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:13.515709 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs podName:4f03a4a6-921a-4b9c-9bba-5de2850a8b70 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:21.515697132 +0000 UTC m=+50.458266033 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs") pod "router-default-6b49ccfd44-hkkz2" (UID: "4f03a4a6-921a-4b9c-9bba-5de2850a8b70") : secret "router-metrics-certs-default" not found Sep 09 02:26:13.515842 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:13.515758 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:13.515909 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:13.515890 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle podName:4f03a4a6-921a-4b9c-9bba-5de2850a8b70 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:21.51587726 +0000 UTC m=+50.458446147 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle") pod "router-default-6b49ccfd44-hkkz2" (UID: "4f03a4a6-921a-4b9c-9bba-5de2850a8b70") : configmap references non-existent config key: service-ca.crt Sep 09 02:26:13.616574 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:13.616537 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:13.616720 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:13.616711 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 02:26:13.616777 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:13.616766 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls podName:5f61825a-519f-42c0-8ab0-3e5fcbd37b19 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:21.616750521 +0000 UTC m=+50.559319417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-xmrb5" (UID: "5f61825a-519f-42c0-8ab0-3e5fcbd37b19") : secret "cluster-monitoring-operator-tls" not found Sep 09 02:26:15.818704 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.818666 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-l47l7" event={"ID":"28b8bb88-2eef-4b7a-bd55-16549de8228f","Type":"ContainerStarted","Data":"2ff644e9ee978fd36b337b069e909b7231d6d127144e5f2d2439e0d5756ad8bb"} Sep 09 02:26:15.820709 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.820680 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" event={"ID":"3018c2e4-70dc-4421-8bb0-540126c40bd5","Type":"ContainerStarted","Data":"1fd62024aaa3065c1ccc8fba26ae89f558f911ae9cce7bbb221bd4b035157e76"} Sep 09 02:26:15.823032 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.823009 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" event={"ID":"15536224-40d3-43d7-b5df-c2a44121f29d","Type":"ContainerStarted","Data":"6519a76b42640c0001fd9d57f18ecaf8c2289e2b3399986c6369086d9ae2118e"} Sep 09 02:26:15.824737 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.824714 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" event={"ID":"1183493c-bda9-4396-858e-ee95549d8a22","Type":"ContainerStarted","Data":"5d68a8aec22f7ff1ef49596a217e6c94d9539deccb622c3f56a6164871561b60"} Sep 09 02:26:15.827004 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.826724 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-92stt" event={"ID":"f73d14e5-2b62-4cb1-962d-c2b4b74446f7","Type":"ContainerStarted","Data":"6e1307690f95c14ec7a6c4ae780cc2717f6fc1b76d48f894aabda8c6e77565d9"} Sep 09 02:26:15.828652 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.828610 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-225b5" event={"ID":"6ff5de1e-f270-439b-a4e0-3bb1394ecb79","Type":"ContainerStarted","Data":"4a5e7dac46c8f5bb79b81fcef2b1f3f982c5dce4d26945f06e9c85e6727f5f02"} Sep 09 02:26:15.830444 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.830423 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/0.log" Sep 09 02:26:15.830546 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.830458 2577 generic.go:356] "Generic (PLEG): container finished" podID="58860e10-d2bc-4a1a-8b98-20d772421b9f" containerID="3f70ef106359516b402548fc25b8fde4093a33786eb027dfe2394f9735b4c3c4" exitCode=255 Sep 09 02:26:15.830546 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.830525 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" event={"ID":"58860e10-d2bc-4a1a-8b98-20d772421b9f","Type":"ContainerDied","Data":"3f70ef106359516b402548fc25b8fde4093a33786eb027dfe2394f9735b4c3c4"} Sep 09 02:26:15.830834 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.830816 2577 scope.go:117] "RemoveContainer" containerID="3f70ef106359516b402548fc25b8fde4093a33786eb027dfe2394f9735b4c3c4" Sep 09 02:26:15.834050 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.833984 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-l47l7" podStartSLOduration=3.229835552 podStartE2EDuration="10.83396723s" totalImagesPullingTime="7.604131678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:15.832406269 +0000 UTC m=+44.774975182" watchObservedRunningTime="2026-09-09 02:26:15.83396723 +0000 UTC m=+44.776536140" Sep 09 02:26:15.836393 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.836373 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-97k2r" event={"ID":"b6b7de4b-5150-40f8-9ef4-f6c50994bb97","Type":"ContainerStarted","Data":"1ff3f651ad7ec11f8eadcc7191cb7e77981ca65d1aba29d876dff8adb53dfb09"} Sep 09 02:26:15.864912 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.864866 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" podStartSLOduration=3.519032455 podStartE2EDuration="10.864847998s" totalImagesPullingTime="7.345815543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:15.86409484 +0000 UTC m=+44.806663753" watchObservedRunningTime="2026-09-09 02:26:15.864847998 +0000 UTC m=+44.807416909" Sep 09 02:26:15.880482 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.880438 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-92stt" podStartSLOduration=2.56655332 podStartE2EDuration="11.880417924s" totalImagesPullingTime="9.313864604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:15.879166477 +0000 UTC m=+44.821735390" watchObservedRunningTime="2026-09-09 02:26:15.880417924 +0000 UTC m=+44.822986833" Sep 09 02:26:15.919443 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.919390 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" podStartSLOduration=3.575055762 podStartE2EDuration="10.919370703s" totalImagesPullingTime="7.344314941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:15.918611088 +0000 UTC m=+44.861179999" watchObservedRunningTime="2026-09-09 02:26:15.919370703 +0000 UTC m=+44.861939615" Sep 09 02:26:15.919901 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.919870 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" podStartSLOduration=3.570705653 podStartE2EDuration="10.919862299s" totalImagesPullingTime="7.349156646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:15.900221282 +0000 UTC m=+44.842790193" watchObservedRunningTime="2026-09-09 02:26:15.919862299 +0000 UTC m=+44.862431188" Sep 09 02:26:15.920134 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.920057 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:15.920134 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.920106 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:15.935273 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.934833 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-225b5" podStartSLOduration=3.59353355 podStartE2EDuration="10.934814774s" totalImagesPullingTime="7.341281224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:15.933948302 +0000 UTC m=+44.876517213" watchObservedRunningTime="2026-09-09 02:26:15.934814774 +0000 UTC m=+44.877383685" Sep 09 02:26:15.961591 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:15.961490 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-97k2r" podStartSLOduration=14.983375106 podStartE2EDuration="44.961475457s" totalImagesPullingTime="29.978100351s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:25:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:15.961161261 +0000 UTC m=+44.903730172" watchObservedRunningTime="2026-09-09 02:26:15.961475457 +0000 UTC m=+44.904044367" Sep 09 02:26:16.082331 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.082263 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:16.181817 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.181786 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:16.282059 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.282024 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:16.338270 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.338181 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:16.501288 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.501244 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:16.551070 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.550998 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-l4gcf"] Sep 09 02:26:16.572148 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.572113 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l4gcf"] Sep 09 02:26:16.572307 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.572232 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.575313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.575262 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 02:26:16.575313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.575307 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8wjcz\"" Sep 09 02:26:16.575521 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.575362 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 02:26:16.576254 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.576196 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:16.648064 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.647988 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlk2q\" (UniqueName: \"kubernetes.io/projected/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-api-access-tlk2q\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.648064 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.648058 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdbde3d2-c31b-45fb-af40-a4c2afea6265-data-volume\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.648283 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.648154 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.648283 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.648194 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.648283 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.648245 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fdbde3d2-c31b-45fb-af40-a4c2afea6265-crio-socket\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.749785 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.749749 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.749785 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.749787 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.749966 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:16.749864 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 02:26:16.749966 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:16.749893 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 02:26:16.749966 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:16.749919 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm podName:fdbde3d2-c31b-45fb-af40-a4c2afea6265 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:17.249900382 +0000 UTC m=+46.192469294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l4gcf" (UID: "fdbde3d2-c31b-45fb-af40-a4c2afea6265") : configmap "kube-rbac-proxy" not found Sep 09 02:26:16.749966 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.749915 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fdbde3d2-c31b-45fb-af40-a4c2afea6265-crio-socket\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.749966 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:16.749946 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls podName:fdbde3d2-c31b-45fb-af40-a4c2afea6265 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:17.249928353 +0000 UTC m=+46.192497242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4gcf" (UID: "fdbde3d2-c31b-45fb-af40-a4c2afea6265") : secret "insights-runtime-extractor-tls" not found Sep 09 02:26:16.750138 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.749988 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fdbde3d2-c31b-45fb-af40-a4c2afea6265-crio-socket\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.750138 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.750051 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlk2q\" (UniqueName: \"kubernetes.io/projected/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-api-access-tlk2q\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.750138 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.750114 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdbde3d2-c31b-45fb-af40-a4c2afea6265-data-volume\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.750516 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.750499 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fdbde3d2-c31b-45fb-af40-a4c2afea6265-data-volume\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.763903 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.763872 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlk2q\" (UniqueName: \"kubernetes.io/projected/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-api-access-tlk2q\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:16.824668 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.824612 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:16.846197 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.846163 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/1.log" Sep 09 02:26:16.846662 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.846623 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/0.log" Sep 09 02:26:16.846793 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.846677 2577 generic.go:356] "Generic (PLEG): container finished" podID="58860e10-d2bc-4a1a-8b98-20d772421b9f" containerID="85c50d2c02c370cc2fcc5ad9dde18e11780efdc1190a96316df4ac365e721b0d" exitCode=255 Sep 09 02:26:16.846793 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.846734 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" event={"ID":"58860e10-d2bc-4a1a-8b98-20d772421b9f","Type":"ContainerDied","Data":"85c50d2c02c370cc2fcc5ad9dde18e11780efdc1190a96316df4ac365e721b0d"} Sep 09 02:26:16.846886 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.846838 2577 scope.go:117] "RemoveContainer" containerID="3f70ef106359516b402548fc25b8fde4093a33786eb027dfe2394f9735b4c3c4" Sep 09 02:26:16.847039 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.846998 2577 scope.go:117] "RemoveContainer" containerID="85c50d2c02c370cc2fcc5ad9dde18e11780efdc1190a96316df4ac365e721b0d" Sep 09 02:26:16.847219 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:16.847193 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-d4vlb_openshift-console-operator(58860e10-d2bc-4a1a-8b98-20d772421b9f)\"" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" podUID="58860e10-d2bc-4a1a-8b98-20d772421b9f" Sep 09 02:26:16.885166 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.885143 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:16.942574 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:16.942503 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:17.254827 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:17.254791 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:17.255022 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:17.254832 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:17.255022 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:17.254977 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 02:26:17.255022 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:17.254998 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 02:26:17.255180 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:17.255067 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls podName:fdbde3d2-c31b-45fb-af40-a4c2afea6265 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:18.255040858 +0000 UTC m=+47.197609746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4gcf" (UID: "fdbde3d2-c31b-45fb-af40-a4c2afea6265") : secret "insights-runtime-extractor-tls" not found Sep 09 02:26:17.255180 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:17.255089 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm podName:fdbde3d2-c31b-45fb-af40-a4c2afea6265 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:18.255078674 +0000 UTC m=+47.197647567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l4gcf" (UID: "fdbde3d2-c31b-45fb-af40-a4c2afea6265") : configmap "kube-rbac-proxy" not found Sep 09 02:26:17.492414 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:17.492384 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:17.541937 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:17.541864 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:17.720981 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:17.720949 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:17.851539 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:17.851471 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/1.log" Sep 09 02:26:17.851967 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:17.851788 2577 scope.go:117] "RemoveContainer" containerID="85c50d2c02c370cc2fcc5ad9dde18e11780efdc1190a96316df4ac365e721b0d" Sep 09 02:26:17.851967 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:17.851955 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-d4vlb_openshift-console-operator(58860e10-d2bc-4a1a-8b98-20d772421b9f)\"" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" podUID="58860e10-d2bc-4a1a-8b98-20d772421b9f" Sep 09 02:26:18.194455 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:18.194376 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:18.262910 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:18.262884 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:18.263048 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:18.262912 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:18.263048 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:18.263014 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 02:26:18.263153 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:18.263043 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 02:26:18.263153 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:18.263063 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm podName:fdbde3d2-c31b-45fb-af40-a4c2afea6265 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:20.26305099 +0000 UTC m=+49.205619878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l4gcf" (UID: "fdbde3d2-c31b-45fb-af40-a4c2afea6265") : configmap "kube-rbac-proxy" not found Sep 09 02:26:18.263153 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:18.263114 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls podName:fdbde3d2-c31b-45fb-af40-a4c2afea6265 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:20.26309471 +0000 UTC m=+49.205663613 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4gcf" (UID: "fdbde3d2-c31b-45fb-af40-a4c2afea6265") : secret "insights-runtime-extractor-tls" not found Sep 09 02:26:18.297134 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:18.297108 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:18.331610 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:18.331590 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sffcl_d5d907cf-31d4-42d9-b7e1-69ee4f71c967/dns-node-resolver/0.log" Sep 09 02:26:19.132397 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:19.132367 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t4fn5_2f661248-5853-4ec4-99ef-f03ae698507b/node-ca/0.log" Sep 09 02:26:19.649462 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:19.649427 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:19.698870 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:19.698842 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:19.736715 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:19.736692 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-92stt_f73d14e5-2b62-4cb1-962d-c2b4b74446f7/serve-healthcheck-canary/0.log" Sep 09 02:26:19.756824 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:19.756800 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:19.841445 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:19.841414 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:19.894360 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:19.894330 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:19.994885 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:19.994856 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:20.218898 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:20.218866 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:20.280379 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:20.280309 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:20.280379 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:20.280341 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:20.280535 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:20.280434 2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 02:26:20.280535 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:20.280458 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 02:26:20.280535 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:20.280488 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm podName:fdbde3d2-c31b-45fb-af40-a4c2afea6265 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:24.280475552 +0000 UTC m=+53.223044440 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm") pod "insights-runtime-extractor-l4gcf" (UID: "fdbde3d2-c31b-45fb-af40-a4c2afea6265") : configmap "kube-rbac-proxy" not found Sep 09 02:26:20.280535 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:20.280507 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls podName:fdbde3d2-c31b-45fb-af40-a4c2afea6265 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:24.280493877 +0000 UTC m=+53.223062765 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4gcf" (UID: "fdbde3d2-c31b-45fb-af40-a4c2afea6265") : secret "insights-runtime-extractor-tls" not found Sep 09 02:26:20.333086 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:20.333057 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:20.784994 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:20.784956 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:20.785140 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:20.785119 2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 02:26:20.785202 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:20.785131 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:20.785202 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:20.785198 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls podName:34a35235-50fe-4c00-970b-7d1d13fb59aa nodeName:}" failed. No retries permitted until 2026-09-09 02:26:36.785167331 +0000 UTC m=+65.727736239 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls") pod "dns-default-7nbg2" (UID: "34a35235-50fe-4c00-970b-7d1d13fb59aa") : secret "dns-default-metrics-tls" not found Sep 09 02:26:20.785284 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:20.785259 2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 02:26:20.785284 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:20.785272 2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-c69dff986-kcc28: secret "image-registry-tls" not found Sep 09 02:26:20.785351 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:20.785313 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls podName:e2d68c24-03df-44b2-9bf1-5ef24b24577b nodeName:}" failed. No retries permitted until 2026-09-09 02:26:36.785301111 +0000 UTC m=+65.727869999 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls") pod "image-registry-c69dff986-kcc28" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b") : secret "image-registry-tls" not found Sep 09 02:26:20.943381 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:20.943318 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:21.320652 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:21.320600 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:21.592369 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:21.592271 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:21.592369 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:21.592338 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7hs9h\" (UID: \"1afc5697-33db-4342-8de4-e1145ad105ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:21.592589 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:21.592416 2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 02:26:21.592589 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:21.592457 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:21.592589 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:21.592471 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs podName:4f03a4a6-921a-4b9c-9bba-5de2850a8b70 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:37.592455461 +0000 UTC m=+66.535024348 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs") pod "router-default-6b49ccfd44-hkkz2" (UID: "4f03a4a6-921a-4b9c-9bba-5de2850a8b70") : secret "router-metrics-certs-default" not found Sep 09 02:26:21.592589 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:21.592493 2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 02:26:21.592589 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:21.592553 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls podName:1afc5697-33db-4342-8de4-e1145ad105ef nodeName:}" failed. No retries permitted until 2026-09-09 02:26:37.59253729 +0000 UTC m=+66.535106183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-7hs9h" (UID: "1afc5697-33db-4342-8de4-e1145ad105ef") : secret "samples-operator-tls" not found Sep 09 02:26:21.592589 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:21.592568 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle podName:4f03a4a6-921a-4b9c-9bba-5de2850a8b70 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:37.592562113 +0000 UTC m=+66.535131002 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle") pod "router-default-6b49ccfd44-hkkz2" (UID: "4f03a4a6-921a-4b9c-9bba-5de2850a8b70") : configmap references non-existent config key: service-ca.crt Sep 09 02:26:21.693346 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:21.693310 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:21.693503 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:21.693421 2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 02:26:21.693503 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:21.693472 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls podName:5f61825a-519f-42c0-8ab0-3e5fcbd37b19 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:37.693459423 +0000 UTC m=+66.636028324 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-xmrb5" (UID: "5f61825a-519f-42c0-8ab0-3e5fcbd37b19") : secret "cluster-monitoring-operator-tls" not found Sep 09 02:26:23.260690 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:23.260655 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:24.316261 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:24.316220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:24.316261 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:24.316263 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:24.316740 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:24.316369 2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 02:26:24.316740 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:24.316433 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls podName:fdbde3d2-c31b-45fb-af40-a4c2afea6265 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:32.316416307 +0000 UTC m=+61.258985195 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls") pod "insights-runtime-extractor-l4gcf" (UID: "fdbde3d2-c31b-45fb-af40-a4c2afea6265") : secret "insights-runtime-extractor-tls" not found Sep 09 02:26:24.316813 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:24.316777 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fdbde3d2-c31b-45fb-af40-a4c2afea6265-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:24.803712 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:24.803674 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:25.087243 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:25.087162 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:25.194452 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:25.194417 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:25.288883 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:25.288851 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:25.386714 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:25.386613 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:25.509872 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:25.509830 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:25.919882 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:25.919840 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:25.920053 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:25.919932 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:25.920255 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:25.920242 2577 scope.go:117] "RemoveContainer" containerID="85c50d2c02c370cc2fcc5ad9dde18e11780efdc1190a96316df4ac365e721b0d" Sep 09 02:26:25.920404 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:25.920386 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-d4vlb_openshift-console-operator(58860e10-d2bc-4a1a-8b98-20d772421b9f)\"" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" podUID="58860e10-d2bc-4a1a-8b98-20d772421b9f" Sep 09 02:26:26.510301 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:26.510264 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:26.707824 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:26.707786 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:27.108826 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:27.108792 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:27.305427 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:27.305390 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:27.705945 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:27.705912 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:27.929924 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:27.929896 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:28.500476 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:28.500447 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:28.550195 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:28.550162 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:28.900564 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:28.900479 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:29.106425 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:29.106396 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:29.298403 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:29.298366 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:29.359617 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:29.359591 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:29.529524 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:29.529493 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:30.501100 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:30.501065 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:30.905642 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:30.905561 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:31.006523 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:31.006495 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:31.105094 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:31.105061 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:31.300368 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:31.300335 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:31.505177 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:31.505150 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:32.383024 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:32.382991 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:32.385262 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:32.385232 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fdbde3d2-c31b-45fb-af40-a4c2afea6265-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-l4gcf\" (UID: \"fdbde3d2-c31b-45fb-af40-a4c2afea6265\") " pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:32.485195 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:32.485163 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8wjcz\"" Sep 09 02:26:32.493034 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:32.493014 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-l4gcf" Sep 09 02:26:32.504728 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:32.504688 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:32.759054 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:32.759028 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-78gs8" Sep 09 02:26:32.768155 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:32.768130 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-l4gcf"] Sep 09 02:26:32.770828 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:32.770801 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdbde3d2_c31b_45fb_af40_a4c2afea6265.slice/crio-327cac1c9aa49140b92d18bce94e360df0e873e268d83d87cd9591d4f0314c8a WatchSource:0}: Error finding container 327cac1c9aa49140b92d18bce94e360df0e873e268d83d87cd9591d4f0314c8a: Status 404 returned error can't find the container with id 327cac1c9aa49140b92d18bce94e360df0e873e268d83d87cd9591d4f0314c8a Sep 09 02:26:32.895931 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:32.895902 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l4gcf" event={"ID":"fdbde3d2-c31b-45fb-af40-a4c2afea6265","Type":"ContainerStarted","Data":"ece880cd8e1e09e9d16741faf4b12dbdd7258573cec85105c49ca388041f24c6"} Sep 09 02:26:32.896065 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:32.895953 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l4gcf" event={"ID":"fdbde3d2-c31b-45fb-af40-a4c2afea6265","Type":"ContainerStarted","Data":"327cac1c9aa49140b92d18bce94e360df0e873e268d83d87cd9591d4f0314c8a"} Sep 09 02:26:32.902190 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:32.902167 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:33.109213 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:33.109131 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:33.305344 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:33.305309 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:33.504981 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:33.504948 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:33.900199 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:33.900114 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l4gcf" event={"ID":"fdbde3d2-c31b-45fb-af40-a4c2afea6265","Type":"ContainerStarted","Data":"d80546fadf79d7418591129775b641c4fb875626372760b93f02e882c827ee77"} Sep 09 02:26:34.302266 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:34.302222 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:34.505953 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:34.505925 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:35.103327 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:35.103289 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:35.305778 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:35.305742 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:35.503427 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:35.503393 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:35.913159 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:35.913127 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-l4gcf" event={"ID":"fdbde3d2-c31b-45fb-af40-a4c2afea6265","Type":"ContainerStarted","Data":"9b2c413c268bff2436434d8ff017748d9e7732ce5b1cd11e557af08d2f259117"} Sep 09 02:26:35.929858 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:35.929807 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-l4gcf" podStartSLOduration=17.907267489 podStartE2EDuration="19.929793013s" totalImagesPullingTime="2.022525524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:26:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:35.928017248 +0000 UTC m=+64.870586170" watchObservedRunningTime="2026-09-09 02:26:35.929793013 +0000 UTC m=+64.872361923" Sep 09 02:26:36.215304 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:36.215203 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:36.264502 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:36.264477 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:36.820857 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:36.820818 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:36.821019 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:36.820904 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:36.823132 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:36.823106 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/34a35235-50fe-4c00-970b-7d1d13fb59aa-metrics-tls\") pod \"dns-default-7nbg2\" (UID: \"34a35235-50fe-4c00-970b-7d1d13fb59aa\") " pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:36.823247 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:36.823229 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls\") pod \"image-registry-c69dff986-kcc28\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:37.020054 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.020018 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h655c\"" Sep 09 02:26:37.028038 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.028017 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:37.031639 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.031616 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t86xl\"" Sep 09 02:26:37.040557 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.040540 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:37.344341 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.344250 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c69dff986-kcc28"] Sep 09 02:26:37.346583 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:37.346559 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2d68c24_03df_44b2_9bf1_5ef24b24577b.slice/crio-e7a2c6503c8098c93766e779303fc1a26e7171e5e9272299634e7de1799d4d92 WatchSource:0}: Error finding container e7a2c6503c8098c93766e779303fc1a26e7171e5e9272299634e7de1799d4d92: Status 404 returned error can't find the container with id e7a2c6503c8098c93766e779303fc1a26e7171e5e9272299634e7de1799d4d92 Sep 09 02:26:37.362176 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.362150 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7nbg2"] Sep 09 02:26:37.364674 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:37.364650 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a35235_50fe_4c00_970b_7d1d13fb59aa.slice/crio-7622bed3dc06c4da627c1ca0dcb85308681ee8cf1fb840327b3804bb45963b31 WatchSource:0}: Error finding container 7622bed3dc06c4da627c1ca0dcb85308681ee8cf1fb840327b3804bb45963b31: Status 404 returned error can't find the container with id 7622bed3dc06c4da627c1ca0dcb85308681ee8cf1fb840327b3804bb45963b31 Sep 09 02:26:37.426759 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.426733 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:26:37.426871 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.426832 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:26:37.429213 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.429193 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 02:26:37.429310 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.429233 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 02:26:37.439073 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.439053 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b87ad709-411d-4c4c-81a7-100bf4ac44d5-original-pull-secret\") pod \"global-pull-secret-syncer-rczng\" (UID: \"b87ad709-411d-4c4c-81a7-100bf4ac44d5\") " pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:26:37.439137 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.439090 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab-metrics-certs\") pod \"network-metrics-daemon-szmkq\" (UID: \"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab\") " pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:26:37.527995 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.527962 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w74rv\" (UniqueName: \"kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv\") pod \"network-check-target-4w7rt\" (UID: \"c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016\") " pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:26:37.530385 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.530367 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w74rv\" (UniqueName: \"kubernetes.io/projected/c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016-kube-api-access-w74rv\") pod \"network-check-target-4w7rt\" (UID: \"c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016\") " pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:26:37.576317 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.576290 2577 scope.go:117] "RemoveContainer" containerID="85c50d2c02c370cc2fcc5ad9dde18e11780efdc1190a96316df4ac365e721b0d" Sep 09 02:26:37.628799 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.628775 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7hs9h\" (UID: \"1afc5697-33db-4342-8de4-e1145ad105ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:37.628930 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.628868 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:37.628930 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.628927 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:37.629739 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.629714 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-service-ca-bundle\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:37.631001 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.630980 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1afc5697-33db-4342-8de4-e1145ad105ef-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-7hs9h\" (UID: \"1afc5697-33db-4342-8de4-e1145ad105ef\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:37.631084 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.631024 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f03a4a6-921a-4b9c-9bba-5de2850a8b70-metrics-certs\") pod \"router-default-6b49ccfd44-hkkz2\" (UID: \"4f03a4a6-921a-4b9c-9bba-5de2850a8b70\") " pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:37.694551 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.694525 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rczng" Sep 09 02:26:37.704071 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.704052 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-v2n6b\"" Sep 09 02:26:37.711977 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.711953 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:26:37.717246 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.717227 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-swb8h\"" Sep 09 02:26:37.725860 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.725836 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-szmkq" Sep 09 02:26:37.730301 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.730276 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:37.732950 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.732921 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5f61825a-519f-42c0-8ab0-3e5fcbd37b19-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-xmrb5\" (UID: \"5f61825a-519f-42c0-8ab0-3e5fcbd37b19\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:37.809527 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.809362 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-4r6k2\"" Sep 09 02:26:37.817712 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.817693 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" Sep 09 02:26:37.822306 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.822289 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-dks4w\"" Sep 09 02:26:37.831717 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.831615 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:37.889138 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.888905 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-2nfst\"" Sep 09 02:26:37.897053 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.897017 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" Sep 09 02:26:37.923574 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.923428 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c69dff986-kcc28" event={"ID":"e2d68c24-03df-44b2-9bf1-5ef24b24577b","Type":"ContainerStarted","Data":"3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe"} Sep 09 02:26:37.923574 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.923490 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c69dff986-kcc28" event={"ID":"e2d68c24-03df-44b2-9bf1-5ef24b24577b","Type":"ContainerStarted","Data":"e7a2c6503c8098c93766e779303fc1a26e7171e5e9272299634e7de1799d4d92"} Sep 09 02:26:37.924286 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.924161 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:26:37.925316 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.925291 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7nbg2" event={"ID":"34a35235-50fe-4c00-970b-7d1d13fb59aa","Type":"ContainerStarted","Data":"7622bed3dc06c4da627c1ca0dcb85308681ee8cf1fb840327b3804bb45963b31"} Sep 09 02:26:37.927006 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.926989 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/2.log" Sep 09 02:26:37.927452 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.927438 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/1.log" Sep 09 02:26:37.927498 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.927465 2577 generic.go:356] "Generic (PLEG): container finished" podID="58860e10-d2bc-4a1a-8b98-20d772421b9f" containerID="af1749550e12bd3f00f39627805e4ba0060a81fe1976df454606029c576a9df0" exitCode=255 Sep 09 02:26:37.927498 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.927486 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" event={"ID":"58860e10-d2bc-4a1a-8b98-20d772421b9f","Type":"ContainerDied","Data":"af1749550e12bd3f00f39627805e4ba0060a81fe1976df454606029c576a9df0"} Sep 09 02:26:37.927578 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.927516 2577 scope.go:117] "RemoveContainer" containerID="85c50d2c02c370cc2fcc5ad9dde18e11780efdc1190a96316df4ac365e721b0d" Sep 09 02:26:37.927941 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.927924 2577 scope.go:117] "RemoveContainer" containerID="af1749550e12bd3f00f39627805e4ba0060a81fe1976df454606029c576a9df0" Sep 09 02:26:37.928121 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:37.928099 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-d4vlb_openshift-console-operator(58860e10-d2bc-4a1a-8b98-20d772421b9f)\"" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" podUID="58860e10-d2bc-4a1a-8b98-20d772421b9f" Sep 09 02:26:37.943710 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.943331 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c69dff986-kcc28" podStartSLOduration=66.943309374 podStartE2EDuration="1m6.943309374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:25:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:37.941649662 +0000 UTC m=+66.884218564" watchObservedRunningTime="2026-09-09 02:26:37.943309374 +0000 UTC m=+66.885878286" Sep 09 02:26:37.970282 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.970245 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rczng"] Sep 09 02:26:37.996218 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:37.996167 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4w7rt"] Sep 09 02:26:37.999189 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:37.999160 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7aaf17f_dc8e_4c1c_a4d2_5bcc64cb4016.slice/crio-fb201bca9bb88843e3af3daff45d49e9de7424daa0c5c32c9583dc7d5d010730 WatchSource:0}: Error finding container fb201bca9bb88843e3af3daff45d49e9de7424daa0c5c32c9583dc7d5d010730: Status 404 returned error can't find the container with id fb201bca9bb88843e3af3daff45d49e9de7424daa0c5c32c9583dc7d5d010730 Sep 09 02:26:38.031789 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.031759 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-szmkq"] Sep 09 02:26:38.035501 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:38.035469 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f4a14b_cdf9_42b2_97a8_a0f3323cf0ab.slice/crio-5722daa97c59183aeeacb63edb3d85f57c384519f8e449f3cac3ff5b6690d744 WatchSource:0}: Error finding container 5722daa97c59183aeeacb63edb3d85f57c384519f8e449f3cac3ff5b6690d744: Status 404 returned error can't find the container with id 5722daa97c59183aeeacb63edb3d85f57c384519f8e449f3cac3ff5b6690d744 Sep 09 02:26:38.121303 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.121180 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h"] Sep 09 02:26:38.154319 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.154295 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6b49ccfd44-hkkz2"] Sep 09 02:26:38.158321 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:38.158290 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f03a4a6_921a_4b9c_9bba_5de2850a8b70.slice/crio-8ad5109adb0e3e787a9c9592b0293ea11871c0210a85b353edb48e85bd89f471 WatchSource:0}: Error finding container 8ad5109adb0e3e787a9c9592b0293ea11871c0210a85b353edb48e85bd89f471: Status 404 returned error can't find the container with id 8ad5109adb0e3e787a9c9592b0293ea11871c0210a85b353edb48e85bd89f471 Sep 09 02:26:38.210983 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.210917 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5"] Sep 09 02:26:38.214198 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:38.214167 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f61825a_519f_42c0_8ab0_3e5fcbd37b19.slice/crio-3999c2f5f63e200f661c15026a7ee6829836afabe22fc39e64a509a640e37ea5 WatchSource:0}: Error finding container 3999c2f5f63e200f661c15026a7ee6829836afabe22fc39e64a509a640e37ea5: Status 404 returned error can't find the container with id 3999c2f5f63e200f661c15026a7ee6829836afabe22fc39e64a509a640e37ea5 Sep 09 02:26:38.936183 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.936035 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/2.log" Sep 09 02:26:38.938165 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.938059 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-szmkq" event={"ID":"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab","Type":"ContainerStarted","Data":"5722daa97c59183aeeacb63edb3d85f57c384519f8e449f3cac3ff5b6690d744"} Sep 09 02:26:38.941643 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.941546 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" event={"ID":"4f03a4a6-921a-4b9c-9bba-5de2850a8b70","Type":"ContainerStarted","Data":"c4027419fda1754d568307aeff9a6f95cc503188452ae3247f0db8a2309504c8"} Sep 09 02:26:38.941643 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.941597 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" event={"ID":"4f03a4a6-921a-4b9c-9bba-5de2850a8b70","Type":"ContainerStarted","Data":"8ad5109adb0e3e787a9c9592b0293ea11871c0210a85b353edb48e85bd89f471"} Sep 09 02:26:38.944227 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.944144 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" event={"ID":"1afc5697-33db-4342-8de4-e1145ad105ef","Type":"ContainerStarted","Data":"23af59ba0e47f3053bc66d008a8b7e0d2d21f2b1dcc7a11a1340f232d9ab7937"} Sep 09 02:26:38.948907 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.948361 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4w7rt" event={"ID":"c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016","Type":"ContainerStarted","Data":"5d1387179ef1b3e4d461dfaeeb5267e27bbc2710ff03a9b4e7cd215457c2d1c6"} Sep 09 02:26:38.948907 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.948403 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4w7rt" event={"ID":"c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016","Type":"ContainerStarted","Data":"fb201bca9bb88843e3af3daff45d49e9de7424daa0c5c32c9583dc7d5d010730"} Sep 09 02:26:38.948907 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.948736 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:26:38.950680 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.950613 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" event={"ID":"5f61825a-519f-42c0-8ab0-3e5fcbd37b19","Type":"ContainerStarted","Data":"3999c2f5f63e200f661c15026a7ee6829836afabe22fc39e64a509a640e37ea5"} Sep 09 02:26:38.952310 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.952260 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rczng" event={"ID":"b87ad709-411d-4c4c-81a7-100bf4ac44d5","Type":"ContainerStarted","Data":"58e5cd2ea77aa447c9b080531fff1b592c5d09d95f9040d9c3e16eee0c6148af"} Sep 09 02:26:38.962031 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.961551 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" podStartSLOduration=33.961533855 podStartE2EDuration="33.961533855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:38.959353412 +0000 UTC m=+67.901922305" watchObservedRunningTime="2026-09-09 02:26:38.961533855 +0000 UTC m=+67.904102767" Sep 09 02:26:38.974215 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:38.973787 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4w7rt" podStartSLOduration=67.973766255 podStartE2EDuration="1m7.973766255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:25:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:38.972390866 +0000 UTC m=+67.914959778" watchObservedRunningTime="2026-09-09 02:26:38.973766255 +0000 UTC m=+67.916335167" Sep 09 02:26:39.832641 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:39.832592 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:39.836145 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:39.836118 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:39.939967 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:39.939925 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:39.956349 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:39.956320 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:39.957781 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:39.957755 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6b49ccfd44-hkkz2" Sep 09 02:26:41.011963 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.010779 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb"] Sep 09 02:26:41.013660 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.013471 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:41.017231 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.016259 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 09 02:26:41.017231 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.016501 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 09 02:26:41.017231 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.016804 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 09 02:26:41.017799 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.017584 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 09 02:26:41.046082 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.045932 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb"] Sep 09 02:26:41.049381 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.049347 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:41.164826 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.164745 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:41.168882 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.168851 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/eaf118e9-17cb-4e40-ab0f-cdd8cfebf870-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b7c8cc8fd-mhskb\" (UID: \"eaf118e9-17cb-4e40-ab0f-cdd8cfebf870\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:41.169022 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.168907 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5ws9\" (UniqueName: \"kubernetes.io/projected/eaf118e9-17cb-4e40-ab0f-cdd8cfebf870-kube-api-access-k5ws9\") pod \"klusterlet-addon-workmgr-7b7c8cc8fd-mhskb\" (UID: \"eaf118e9-17cb-4e40-ab0f-cdd8cfebf870\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:41.169022 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.168969 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eaf118e9-17cb-4e40-ab0f-cdd8cfebf870-tmp\") pod \"klusterlet-addon-workmgr-7b7c8cc8fd-mhskb\" (UID: \"eaf118e9-17cb-4e40-ab0f-cdd8cfebf870\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:41.255725 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.255417 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:41.269910 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.269764 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5ws9\" (UniqueName: \"kubernetes.io/projected/eaf118e9-17cb-4e40-ab0f-cdd8cfebf870-kube-api-access-k5ws9\") pod \"klusterlet-addon-workmgr-7b7c8cc8fd-mhskb\" (UID: \"eaf118e9-17cb-4e40-ab0f-cdd8cfebf870\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:41.269910 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.269845 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eaf118e9-17cb-4e40-ab0f-cdd8cfebf870-tmp\") pod \"klusterlet-addon-workmgr-7b7c8cc8fd-mhskb\" (UID: \"eaf118e9-17cb-4e40-ab0f-cdd8cfebf870\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:41.269910 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.269910 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/eaf118e9-17cb-4e40-ab0f-cdd8cfebf870-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b7c8cc8fd-mhskb\" (UID: \"eaf118e9-17cb-4e40-ab0f-cdd8cfebf870\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:41.270710 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.270678 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eaf118e9-17cb-4e40-ab0f-cdd8cfebf870-tmp\") pod \"klusterlet-addon-workmgr-7b7c8cc8fd-mhskb\" (UID: \"eaf118e9-17cb-4e40-ab0f-cdd8cfebf870\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:41.273052 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.273029 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/eaf118e9-17cb-4e40-ab0f-cdd8cfebf870-klusterlet-config\") pod \"klusterlet-addon-workmgr-7b7c8cc8fd-mhskb\" (UID: \"eaf118e9-17cb-4e40-ab0f-cdd8cfebf870\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:41.282177 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.282153 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5ws9\" (UniqueName: \"kubernetes.io/projected/eaf118e9-17cb-4e40-ab0f-cdd8cfebf870-kube-api-access-k5ws9\") pod \"klusterlet-addon-workmgr-7b7c8cc8fd-mhskb\" (UID: \"eaf118e9-17cb-4e40-ab0f-cdd8cfebf870\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:41.329480 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.329442 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:41.362026 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.361991 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:41.455801 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.455725 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:41.496446 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.496410 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj"] Sep 09 02:26:41.498467 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.498437 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj" Sep 09 02:26:41.500868 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.500845 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 09 02:26:41.501020 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.500849 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-pktms\"" Sep 09 02:26:41.508297 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.508273 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj"] Sep 09 02:26:41.523531 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.523501 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:41.573433 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.573403 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1624d2d9-ca8a-46b7-ac6c-b72663f1d49c-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-zbmwj\" (UID: \"1624d2d9-ca8a-46b7-ac6c-b72663f1d49c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj" Sep 09 02:26:41.673865 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.673825 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1624d2d9-ca8a-46b7-ac6c-b72663f1d49c-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-zbmwj\" (UID: \"1624d2d9-ca8a-46b7-ac6c-b72663f1d49c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj" Sep 09 02:26:41.674041 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:41.673959 2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 09 02:26:41.674041 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:41.674032 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1624d2d9-ca8a-46b7-ac6c-b72663f1d49c-tls-certificates podName:1624d2d9-ca8a-46b7-ac6c-b72663f1d49c nodeName:}" failed. No retries permitted until 2026-09-09 02:26:42.174012648 +0000 UTC m=+71.116581550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/1624d2d9-ca8a-46b7-ac6c-b72663f1d49c-tls-certificates") pod "prometheus-operator-admission-webhook-78f599db79-zbmwj" (UID: "1624d2d9-ca8a-46b7-ac6c-b72663f1d49c") : secret "prometheus-operator-admission-webhook-tls" not found Sep 09 02:26:41.755042 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.755002 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:41.975804 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.975770 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7nbg2" event={"ID":"34a35235-50fe-4c00-970b-7d1d13fb59aa","Type":"ContainerStarted","Data":"e0fa6e4efcad64444d3392979a0c96d134276e86c07aa594a211379e23c56b13"} Sep 09 02:26:41.977619 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.977588 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" event={"ID":"5f61825a-519f-42c0-8ab0-3e5fcbd37b19","Type":"ContainerStarted","Data":"0aba9c33a15926af3491f0f140825d0de19c24d07604d08ffdf39cd06805316d"} Sep 09 02:26:41.979320 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.979294 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-szmkq" event={"ID":"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab","Type":"ContainerStarted","Data":"b5421bce93fc77e60a88d60e7b85b396feec7eed9ac92d6d2438d5eaeeaf39b2"} Sep 09 02:26:41.980947 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.980920 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" event={"ID":"1afc5697-33db-4342-8de4-e1145ad105ef","Type":"ContainerStarted","Data":"0369901371f5feb481b67f4ffc1e838caac4e6fef8b195c7f423c9219c67f54e"} Sep 09 02:26:41.993801 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:41.993747 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-xmrb5" podStartSLOduration=34.324298689 podStartE2EDuration="36.993729048s" totalImagesPullingTime="2.669430359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:41.993017236 +0000 UTC m=+70.935586148" watchObservedRunningTime="2026-09-09 02:26:41.993729048 +0000 UTC m=+70.936297959" Sep 09 02:26:42.178555 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:42.178443 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1624d2d9-ca8a-46b7-ac6c-b72663f1d49c-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-zbmwj\" (UID: \"1624d2d9-ca8a-46b7-ac6c-b72663f1d49c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj" Sep 09 02:26:42.181420 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:42.181391 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1624d2d9-ca8a-46b7-ac6c-b72663f1d49c-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-zbmwj\" (UID: \"1624d2d9-ca8a-46b7-ac6c-b72663f1d49c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj" Sep 09 02:26:42.409594 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:42.409557 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj" Sep 09 02:26:42.763104 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:42.763046 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj"] Sep 09 02:26:42.764667 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:42.764622 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb"] Sep 09 02:26:42.765893 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:42.765861 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1624d2d9_ca8a_46b7_ac6c_b72663f1d49c.slice/crio-400cec2879b3cddfdc6cde86af2384b716c50c30d396e2d6de211cb486af3bd2 WatchSource:0}: Error finding container 400cec2879b3cddfdc6cde86af2384b716c50c30d396e2d6de211cb486af3bd2: Status 404 returned error can't find the container with id 400cec2879b3cddfdc6cde86af2384b716c50c30d396e2d6de211cb486af3bd2 Sep 09 02:26:42.766414 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:42.766381 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaf118e9_17cb_4e40_ab0f_cdd8cfebf870.slice/crio-fb3de9d80a5bcc47c883d54eea377d07cfeed93be9a7a37724e588f58fa11ca8 WatchSource:0}: Error finding container fb3de9d80a5bcc47c883d54eea377d07cfeed93be9a7a37724e588f58fa11ca8: Status 404 returned error can't find the container with id fb3de9d80a5bcc47c883d54eea377d07cfeed93be9a7a37724e588f58fa11ca8 Sep 09 02:26:42.995165 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:42.995120 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7nbg2" event={"ID":"34a35235-50fe-4c00-970b-7d1d13fb59aa","Type":"ContainerStarted","Data":"263c579e87ab0ea509007d5a849801802e7b8df0a9d47634faa8f389929a5914"} Sep 09 02:26:42.995365 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:42.995226 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:42.997102 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:42.997072 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-szmkq" event={"ID":"06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab","Type":"ContainerStarted","Data":"c2796485074ad92b913977fbe04d90d4eb81bbfb6cfaeac372ff26161a91ce92"} Sep 09 02:26:42.998605 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:42.998577 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rczng" event={"ID":"b87ad709-411d-4c4c-81a7-100bf4ac44d5","Type":"ContainerStarted","Data":"149a013a33a16ff91915129493116c6c81e8a836438c860291a7865da3a27e43"} Sep 09 02:26:42.999787 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:42.999765 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj" event={"ID":"1624d2d9-ca8a-46b7-ac6c-b72663f1d49c","Type":"ContainerStarted","Data":"400cec2879b3cddfdc6cde86af2384b716c50c30d396e2d6de211cb486af3bd2"} Sep 09 02:26:43.001446 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:43.001417 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" event={"ID":"1afc5697-33db-4342-8de4-e1145ad105ef","Type":"ContainerStarted","Data":"45d3f823017704df6e6506f1fd5b3cc4ed7b47503a240c4ec545dedc4a7b00d5"} Sep 09 02:26:43.002602 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:43.002580 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" event={"ID":"eaf118e9-17cb-4e40-ab0f-cdd8cfebf870","Type":"ContainerStarted","Data":"fb3de9d80a5bcc47c883d54eea377d07cfeed93be9a7a37724e588f58fa11ca8"} Sep 09 02:26:43.011579 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:43.011536 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7nbg2" podStartSLOduration=35.497500793 podStartE2EDuration="39.011522893s" totalImagesPullingTime="3.5140221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:43.010608039 +0000 UTC m=+71.953176949" watchObservedRunningTime="2026-09-09 02:26:43.011522893 +0000 UTC m=+71.954091803" Sep 09 02:26:43.026091 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:43.025999 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-szmkq" podStartSLOduration=69.181220848 podStartE2EDuration="1m12.025982716s" totalImagesPullingTime="2.844761868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:43.025758175 +0000 UTC m=+71.968327088" watchObservedRunningTime="2026-09-09 02:26:43.025982716 +0000 UTC m=+71.968551626" Sep 09 02:26:43.043021 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:43.042964 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rczng" podStartSLOduration=66.44415995 podStartE2EDuration="1m11.042948878s" totalImagesPullingTime="4.598788928s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:43.042555989 +0000 UTC m=+71.985124899" watchObservedRunningTime="2026-09-09 02:26:43.042948878 +0000 UTC m=+71.985517787" Sep 09 02:26:43.057901 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:43.057836 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-7hs9h" podStartSLOduration=35.365297425 podStartE2EDuration="38.057817017s" totalImagesPullingTime="2.692519592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:43.057607378 +0000 UTC m=+72.000176287" watchObservedRunningTime="2026-09-09 02:26:43.057817017 +0000 UTC m=+72.000385928" Sep 09 02:26:43.146419 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:43.146385 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:43.247823 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:43.247779 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:44.009084 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.009042 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj" event={"ID":"1624d2d9-ca8a-46b7-ac6c-b72663f1d49c","Type":"ContainerStarted","Data":"523d1e905fc30b9f7ffbee7fe7e08d4a5a12bf2b8fa43638203afe606596a76c"} Sep 09 02:26:44.009669 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.009597 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj" Sep 09 02:26:44.015323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.015295 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj" Sep 09 02:26:44.024769 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.024707 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-zbmwj" podStartSLOduration=2.073504329 podStartE2EDuration="3.024685519s" totalImagesPullingTime="951.18119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:44.023700698 +0000 UTC m=+72.966269612" watchObservedRunningTime="2026-09-09 02:26:44.024685519 +0000 UTC m=+72.967254430" Sep 09 02:26:44.520466 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.520429 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:44.554735 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.554701 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-9v69n"] Sep 09 02:26:44.578717 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.578683 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-9v69n"] Sep 09 02:26:44.578891 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.578787 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:44.582848 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.582600 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 09 02:26:44.582848 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.582653 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 09 02:26:44.582848 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.582600 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 02:26:44.583112 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.582953 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-fv9fb\"" Sep 09 02:26:44.598131 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.598082 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:44.705720 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.705666 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:44.705936 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.705732 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:44.705936 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.705802 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj7cr\" (UniqueName: \"kubernetes.io/projected/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-kube-api-access-zj7cr\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:44.705936 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.705886 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:44.807106 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.807001 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:44.807106 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.807085 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:44.807327 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:44.807192 2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Sep 09 02:26:44.807327 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:44.807254 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-prometheus-operator-tls podName:19dd1e0c-909e-4ef8-8c92-46e4f98024e3 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:45.307236173 +0000 UTC m=+74.249805062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-prometheus-operator-tls") pod "prometheus-operator-6f4946db56-9v69n" (UID: "19dd1e0c-909e-4ef8-8c92-46e4f98024e3") : secret "prometheus-operator-tls" not found Sep 09 02:26:44.807327 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.807255 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:44.807327 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.807302 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj7cr\" (UniqueName: \"kubernetes.io/projected/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-kube-api-access-zj7cr\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:44.808014 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.807987 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:44.809720 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.809696 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:44.815233 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:44.815209 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj7cr\" (UniqueName: \"kubernetes.io/projected/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-kube-api-access-zj7cr\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:45.117974 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:45.117891 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:45.312893 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:45.312856 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:45.315548 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:45.315525 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/19dd1e0c-909e-4ef8-8c92-46e4f98024e3-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-9v69n\" (UID: \"19dd1e0c-909e-4ef8-8c92-46e4f98024e3\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:45.491142 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:45.491107 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" Sep 09 02:26:45.779953 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:45.779927 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-9v69n"] Sep 09 02:26:45.783131 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:45.783098 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19dd1e0c_909e_4ef8_8c92_46e4f98024e3.slice/crio-91652ada774956b498c4e26a3c8be538f750a919db6e095d8e5707f4186ae9fd WatchSource:0}: Error finding container 91652ada774956b498c4e26a3c8be538f750a919db6e095d8e5707f4186ae9fd: Status 404 returned error can't find the container with id 91652ada774956b498c4e26a3c8be538f750a919db6e095d8e5707f4186ae9fd Sep 09 02:26:45.919461 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:45.919365 2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:45.919461 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:45.919430 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:26:45.919859 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:45.919842 2577 scope.go:117] "RemoveContainer" containerID="af1749550e12bd3f00f39627805e4ba0060a81fe1976df454606029c576a9df0" Sep 09 02:26:45.920051 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:45.920017 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-d4vlb_openshift-console-operator(58860e10-d2bc-4a1a-8b98-20d772421b9f)\"" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" podUID="58860e10-d2bc-4a1a-8b98-20d772421b9f" Sep 09 02:26:46.019669 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:46.019616 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" event={"ID":"19dd1e0c-909e-4ef8-8c92-46e4f98024e3","Type":"ContainerStarted","Data":"91652ada774956b498c4e26a3c8be538f750a919db6e095d8e5707f4186ae9fd"} Sep 09 02:26:47.150078 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:47.150011 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:47.212610 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:47.212566 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:48.887981 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:48.887899 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:49.033664 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:49.033613 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" event={"ID":"eaf118e9-17cb-4e40-ab0f-cdd8cfebf870","Type":"ContainerStarted","Data":"96ec6609d301848e8535e482ffa1cbc01bb913945ed20e789dd63b5b7919840e"} Sep 09 02:26:49.033852 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:49.033777 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:49.035747 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:49.035718 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" event={"ID":"19dd1e0c-909e-4ef8-8c92-46e4f98024e3","Type":"ContainerStarted","Data":"f3a992a34cd0976d22baa982991bda8949d3d4203e54db64aa69fa6df87182b5"} Sep 09 02:26:49.035869 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:49.035768 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" event={"ID":"19dd1e0c-909e-4ef8-8c92-46e4f98024e3","Type":"ContainerStarted","Data":"fa15a94f6f422ffa234505b7388f3c45ac4f3a12262ad29c94dcf175fc50c48e"} Sep 09 02:26:49.035869 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:49.035818 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" Sep 09 02:26:49.048324 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:49.048276 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7b7c8cc8fd-mhskb" podStartSLOduration=3.511294019 podStartE2EDuration="9.048261532s" totalImagesPullingTime="5.536967513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:49.047377994 +0000 UTC m=+77.989946885" watchObservedRunningTime="2026-09-09 02:26:49.048261532 +0000 UTC m=+77.990830441" Sep 09 02:26:49.079777 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:49.079726 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-9v69n" podStartSLOduration=2.261158943 podStartE2EDuration="5.079711155s" totalImagesPullingTime="2.818552212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:49.079192023 +0000 UTC m=+78.021760935" watchObservedRunningTime="2026-09-09 02:26:49.079711155 +0000 UTC m=+78.022280079" Sep 09 02:26:50.011961 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:50.011929 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7nbg2" Sep 09 02:26:50.682914 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:50.682883 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:50.802106 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:50.802076 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:50.894270 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:50.894237 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:50.908087 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:50.908056 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-k4wbl"] Sep 09 02:26:50.911031 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:50.911017 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:50.913332 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:50.913310 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 02:26:50.913332 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:50.913329 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7c5x9\"" Sep 09 02:26:50.913332 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:50.913308 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 02:26:50.913665 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:50.913646 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 02:26:50.968463 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:50.968390 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:51.063778 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.063744 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.064159 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.063788 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b60dbcc-a928-483e-805e-6ea855cc4959-sys\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.064159 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.063802 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b60dbcc-a928-483e-805e-6ea855cc4959-root\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.064159 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.063858 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-tls\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.064159 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.063899 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b60dbcc-a928-483e-805e-6ea855cc4959-metrics-client-ca\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.064159 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.063932 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-accelerators-collector-config\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.064159 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.063955 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsv26\" (UniqueName: \"kubernetes.io/projected/8b60dbcc-a928-483e-805e-6ea855cc4959-kube-api-access-rsv26\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.064159 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.063993 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-textfile\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.064159 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.064009 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-wtmp\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.164517 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.164484 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.164687 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.164530 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b60dbcc-a928-483e-805e-6ea855cc4959-sys\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.164687 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.164547 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b60dbcc-a928-483e-805e-6ea855cc4959-root\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.164687 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.164565 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-tls\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.164687 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.164625 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8b60dbcc-a928-483e-805e-6ea855cc4959-root\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.164687 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:51.164670 2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 09 02:26:51.164687 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.164625 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8b60dbcc-a928-483e-805e-6ea855cc4959-sys\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.164687 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.164675 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b60dbcc-a928-483e-805e-6ea855cc4959-metrics-client-ca\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.164983 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:26:51.164721 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-tls podName:8b60dbcc-a928-483e-805e-6ea855cc4959 nodeName:}" failed. No retries permitted until 2026-09-09 02:26:51.664707091 +0000 UTC m=+80.607275978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-tls") pod "node-exporter-k4wbl" (UID: "8b60dbcc-a928-483e-805e-6ea855cc4959") : secret "node-exporter-tls" not found Sep 09 02:26:51.164983 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.164772 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-accelerators-collector-config\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.164983 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.164798 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsv26\" (UniqueName: \"kubernetes.io/projected/8b60dbcc-a928-483e-805e-6ea855cc4959-kube-api-access-rsv26\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.164983 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.164830 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-textfile\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.164983 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.164856 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-wtmp\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.165238 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.165092 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-wtmp\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.165238 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.165198 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-textfile\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.165321 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.165248 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b60dbcc-a928-483e-805e-6ea855cc4959-metrics-client-ca\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.165361 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.165311 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-accelerators-collector-config\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.166825 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.166808 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.175765 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.175744 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsv26\" (UniqueName: \"kubernetes.io/projected/8b60dbcc-a928-483e-805e-6ea855cc4959-kube-api-access-rsv26\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.232614 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.232539 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:51.285866 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.285835 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:51.454263 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.454235 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:51.553892 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.553861 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:51.654053 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.654017 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:51.669186 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.669158 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-tls\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.671643 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.671608 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8b60dbcc-a928-483e-805e-6ea855cc4959-node-exporter-tls\") pod \"node-exporter-k4wbl\" (UID: \"8b60dbcc-a928-483e-805e-6ea855cc4959\") " pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.820093 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.820013 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k4wbl" Sep 09 02:26:51.831351 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:26:51.831326 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b60dbcc_a928_483e_805e_6ea855cc4959.slice/crio-8f72fe0f253632920d1f784ca4df59635fc7386004b8bc547ed2567889184add WatchSource:0}: Error finding container 8f72fe0f253632920d1f784ca4df59635fc7386004b8bc547ed2567889184add: Status 404 returned error can't find the container with id 8f72fe0f253632920d1f784ca4df59635fc7386004b8bc547ed2567889184add Sep 09 02:26:51.887739 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.887699 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:51.957778 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:51.957743 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:52.049112 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:52.049074 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k4wbl" event={"ID":"8b60dbcc-a928-483e-805e-6ea855cc4959","Type":"ContainerStarted","Data":"8f72fe0f253632920d1f784ca4df59635fc7386004b8bc547ed2567889184add"} Sep 09 02:26:52.114785 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:52.114692 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:52.301528 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:52.301489 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:53.055815 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:53.055781 2577 generic.go:356] "Generic (PLEG): container finished" podID="8b60dbcc-a928-483e-805e-6ea855cc4959" containerID="2ed091b08cabc2be4cf5b5ccf4270917ecd52b3c5d76535a8c7f220515dbf555" exitCode=0 Sep 09 02:26:53.055993 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:53.055868 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k4wbl" event={"ID":"8b60dbcc-a928-483e-805e-6ea855cc4959","Type":"ContainerDied","Data":"2ed091b08cabc2be4cf5b5ccf4270917ecd52b3c5d76535a8c7f220515dbf555"} Sep 09 02:26:53.083404 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:53.083371 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:53.863514 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:53.863393 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:53.917072 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:53.917033 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:54.064218 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:54.064182 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k4wbl" event={"ID":"8b60dbcc-a928-483e-805e-6ea855cc4959","Type":"ContainerStarted","Data":"cbc002cf568cd3f03753fcb434b0f358f34d21adaa29d9030b60cd74cbe79686"} Sep 09 02:26:54.064400 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:54.064260 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k4wbl" event={"ID":"8b60dbcc-a928-483e-805e-6ea855cc4959","Type":"ContainerStarted","Data":"6518044fca19a56c93ac491b3bf6c6183c683ebf1b5504a1c450844543f62a88"} Sep 09 02:26:54.088304 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:54.088252 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-k4wbl" podStartSLOduration=3.040299173 podStartE2EDuration="4.088236696s" totalImagesPullingTime="986.776122ms" totalInitContainerRuntime="61.161401ms" isStatefulPod=true podCreationTimestamp="2026-09-09 02:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:26:54.088032615 +0000 UTC m=+83.030601528" watchObservedRunningTime="2026-09-09 02:26:54.088236696 +0000 UTC m=+83.030805607" Sep 09 02:26:54.246466 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:54.246433 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:54.307168 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:54.307140 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:55.686502 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:55.686469 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:55.793328 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:55.793297 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:57.032401 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:57.032356 2577 patch_prober.go:28] interesting pod/image-registry-c69dff986-kcc28 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 02:26:57.032871 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:57.032417 2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-c69dff986-kcc28" podUID="e2d68c24-03df-44b2-9bf1-5ef24b24577b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 02:26:57.108598 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:57.108561 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:58.540188 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:58.540156 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:58.632456 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:58.632421 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:58.689124 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:58.689091 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:59.133139 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:59.133108 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:59.234356 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:59.234323 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:26:59.960902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:26:59.960873 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:27:00.120377 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:00.120341 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:00.182518 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:00.182483 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:00.575312 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:00.575272 2577 scope.go:117] "RemoveContainer" containerID="af1749550e12bd3f00f39627805e4ba0060a81fe1976df454606029c576a9df0" Sep 09 02:27:01.096086 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.096059 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/2.log" Sep 09 02:27:01.096474 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.096155 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" event={"ID":"58860e10-d2bc-4a1a-8b98-20d772421b9f","Type":"ContainerStarted","Data":"fa0919550022a226c511d0fed58a7a43348b8837b4120080a7f88f8d1d56f039"} Sep 09 02:27:01.096474 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.096428 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:27:01.101285 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.101263 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" Sep 09 02:27:01.121688 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.121624 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-d4vlb" podStartSLOduration=48.778945699 podStartE2EDuration="56.121608304s" totalImagesPullingTime="7.342662605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:27:01.120948231 +0000 UTC m=+90.063517154" watchObservedRunningTime="2026-09-09 02:27:01.121608304 +0000 UTC m=+90.064177215" Sep 09 02:27:01.222019 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.221989 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-wrhv7"] Sep 09 02:27:01.224024 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.223997 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-wrhv7" Sep 09 02:27:01.226870 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.226844 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-lmx7p\"" Sep 09 02:27:01.226870 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.226847 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 09 02:27:01.227249 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.227212 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 09 02:27:01.230920 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.230798 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:01.236055 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.236037 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtcnh\" (UniqueName: \"kubernetes.io/projected/3ca0346c-d2df-4fda-9620-4b8837d64de8-kube-api-access-jtcnh\") pod \"downloads-85d98c998f-wrhv7\" (UID: \"3ca0346c-d2df-4fda-9620-4b8837d64de8\") " pod="openshift-console/downloads-85d98c998f-wrhv7" Sep 09 02:27:01.237776 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.237757 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-wrhv7"] Sep 09 02:27:01.294505 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.294463 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:01.336779 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.336746 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtcnh\" (UniqueName: \"kubernetes.io/projected/3ca0346c-d2df-4fda-9620-4b8837d64de8-kube-api-access-jtcnh\") pod \"downloads-85d98c998f-wrhv7\" (UID: \"3ca0346c-d2df-4fda-9620-4b8837d64de8\") " pod="openshift-console/downloads-85d98c998f-wrhv7" Sep 09 02:27:01.346085 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.346049 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtcnh\" (UniqueName: \"kubernetes.io/projected/3ca0346c-d2df-4fda-9620-4b8837d64de8-kube-api-access-jtcnh\") pod \"downloads-85d98c998f-wrhv7\" (UID: \"3ca0346c-d2df-4fda-9620-4b8837d64de8\") " pod="openshift-console/downloads-85d98c998f-wrhv7" Sep 09 02:27:01.394909 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.394823 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:01.517460 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.517418 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:01.533319 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.533291 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-wrhv7" Sep 09 02:27:01.804030 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.803956 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-wrhv7"] Sep 09 02:27:01.807813 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:27:01.807784 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca0346c_d2df_4fda_9620_4b8837d64de8.slice/crio-49339315dd861a3eb2ac7010880edc81e88abd5f3ca3c33e3af086fbaebe620e WatchSource:0}: Error finding container 49339315dd861a3eb2ac7010880edc81e88abd5f3ca3c33e3af086fbaebe620e: Status 404 returned error can't find the container with id 49339315dd861a3eb2ac7010880edc81e88abd5f3ca3c33e3af086fbaebe620e Sep 09 02:27:01.889249 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.889149 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:01.987356 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:01.987322 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:02.102122 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:02.102037 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-wrhv7" event={"ID":"3ca0346c-d2df-4fda-9620-4b8837d64de8","Type":"ContainerStarted","Data":"49339315dd861a3eb2ac7010880edc81e88abd5f3ca3c33e3af086fbaebe620e"} Sep 09 02:27:02.297313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:02.297280 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:02.388990 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:02.388903 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:02.696534 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:02.696457 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:02.897827 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:02.897797 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:03.270842 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:03.270808 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c69dff986-kcc28"] Sep 09 02:27:03.353049 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:03.353019 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:03.454164 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:03.454117 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:03.509299 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:03.509259 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:04.114724 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:04.114677 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:04.171543 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:04.171502 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:04.351292 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:04.351246 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:04.452833 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:04.452750 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:06.413902 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:06.413865 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:06.468447 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:06.468410 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:09.142850 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:09.142767 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:09.248815 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:09.248774 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:09.345722 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:09.345687 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:09.959101 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:09.959009 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4w7rt" Sep 09 02:27:13.416045 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:13.416009 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:13.519211 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:13.519162 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:16.109755 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:16.109712 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:16.158860 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:16.158823 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:18.276786 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:18.276751 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:19.714557 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:19.714524 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:20.187550 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:20.187504 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-wrhv7" event={"ID":"3ca0346c-d2df-4fda-9620-4b8837d64de8","Type":"ContainerStarted","Data":"65dbb83ab643e3f1957e45ddef74bb92f71d2439afafec9acb414fae71f96c08"} Sep 09 02:27:20.187895 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:20.187832 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-wrhv7" Sep 09 02:27:20.207494 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:20.207391 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-wrhv7" Sep 09 02:27:20.208837 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:20.208789 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-wrhv7" podStartSLOduration=1.323858076 podStartE2EDuration="19.208775333s" totalImagesPullingTime="17.884917257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:27:20.207719837 +0000 UTC m=+109.150288748" watchObservedRunningTime="2026-09-09 02:27:20.208775333 +0000 UTC m=+109.151344270" Sep 09 02:27:21.232582 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:21.232543 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:21.332149 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:21.332112 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:22.714619 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:22.714562 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:22.816554 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:22.816503 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:24.001204 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:24.001159 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:24.059001 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:24.058941 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:27.920496 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:27.920457 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:28.022696 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.022659 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:28.300193 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.300150 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-c69dff986-kcc28" podUID="e2d68c24-03df-44b2-9bf1-5ef24b24577b" containerName="registry" containerID="cri-o://3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe" gracePeriod=30 Sep 09 02:27:28.586859 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.586824 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:27:28.686330 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.686296 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxqwj\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-kube-api-access-rxqwj\") pod \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " Sep 09 02:27:28.686538 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.686334 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-bound-sa-token\") pod \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " Sep 09 02:27:28.686538 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.686364 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2d68c24-03df-44b2-9bf1-5ef24b24577b-image-registry-private-configuration\") pod \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " Sep 09 02:27:28.686538 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.686392 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2d68c24-03df-44b2-9bf1-5ef24b24577b-installation-pull-secrets\") pod \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " Sep 09 02:27:28.686750 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.686545 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls\") pod \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " Sep 09 02:27:28.686750 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.686578 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2d68c24-03df-44b2-9bf1-5ef24b24577b-trusted-ca\") pod \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " Sep 09 02:27:28.686750 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.686598 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-certificates\") pod \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " Sep 09 02:27:28.686750 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.686671 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2d68c24-03df-44b2-9bf1-5ef24b24577b-ca-trust-extracted\") pod \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\" (UID: \"e2d68c24-03df-44b2-9bf1-5ef24b24577b\") " Sep 09 02:27:28.687227 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.687150 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2d68c24-03df-44b2-9bf1-5ef24b24577b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e2d68c24-03df-44b2-9bf1-5ef24b24577b" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:27:28.687227 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.687162 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e2d68c24-03df-44b2-9bf1-5ef24b24577b" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:27:28.688896 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.688867 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2d68c24-03df-44b2-9bf1-5ef24b24577b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e2d68c24-03df-44b2-9bf1-5ef24b24577b" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:27:28.689173 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.689148 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e2d68c24-03df-44b2-9bf1-5ef24b24577b" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:27:28.689273 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.689183 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2d68c24-03df-44b2-9bf1-5ef24b24577b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e2d68c24-03df-44b2-9bf1-5ef24b24577b" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:27:28.689273 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.689238 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-kube-api-access-rxqwj" (OuterVolumeSpecName: "kube-api-access-rxqwj") pod "e2d68c24-03df-44b2-9bf1-5ef24b24577b" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b"). InnerVolumeSpecName "kube-api-access-rxqwj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:27:28.689348 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.689276 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e2d68c24-03df-44b2-9bf1-5ef24b24577b" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:27:28.695131 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.695107 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2d68c24-03df-44b2-9bf1-5ef24b24577b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e2d68c24-03df-44b2-9bf1-5ef24b24577b" (UID: "e2d68c24-03df-44b2-9bf1-5ef24b24577b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 02:27:28.787666 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.787609 2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-tls\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:27:28.787666 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.787668 2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e2d68c24-03df-44b2-9bf1-5ef24b24577b-trusted-ca\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:27:28.787866 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.787684 2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e2d68c24-03df-44b2-9bf1-5ef24b24577b-registry-certificates\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:27:28.787866 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.787700 2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e2d68c24-03df-44b2-9bf1-5ef24b24577b-ca-trust-extracted\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:27:28.787866 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.787714 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxqwj\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-kube-api-access-rxqwj\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:27:28.787866 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.787730 2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e2d68c24-03df-44b2-9bf1-5ef24b24577b-bound-sa-token\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:27:28.787866 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.787745 2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e2d68c24-03df-44b2-9bf1-5ef24b24577b-image-registry-private-configuration\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:27:28.787866 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:28.787759 2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e2d68c24-03df-44b2-9bf1-5ef24b24577b-installation-pull-secrets\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:27:29.229498 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:29.229456 2577 generic.go:356] "Generic (PLEG): container finished" podID="e2d68c24-03df-44b2-9bf1-5ef24b24577b" containerID="3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe" exitCode=0 Sep 09 02:27:29.230017 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:29.229522 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c69dff986-kcc28" Sep 09 02:27:29.230017 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:29.229537 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c69dff986-kcc28" event={"ID":"e2d68c24-03df-44b2-9bf1-5ef24b24577b","Type":"ContainerDied","Data":"3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe"} Sep 09 02:27:29.230017 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:29.229615 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c69dff986-kcc28" event={"ID":"e2d68c24-03df-44b2-9bf1-5ef24b24577b","Type":"ContainerDied","Data":"e7a2c6503c8098c93766e779303fc1a26e7171e5e9272299634e7de1799d4d92"} Sep 09 02:27:29.230017 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:29.229691 2577 scope.go:117] "RemoveContainer" containerID="3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe" Sep 09 02:27:29.246580 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:29.246559 2577 scope.go:117] "RemoveContainer" containerID="3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe" Sep 09 02:27:29.246911 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:27:29.246889 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe\": container with ID starting with 3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe not found: ID does not exist" containerID="3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe" Sep 09 02:27:29.246967 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:29.246919 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe"} err="failed to get container status \"3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe\": rpc error: code = NotFound desc = could not find container \"3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe\": container with ID starting with 3675b39dc52c2f371c4e7e7384f4123c386ec405ff5f9562adc18268a84302fe not found: ID does not exist" Sep 09 02:27:29.251185 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:29.251164 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-c69dff986-kcc28"] Sep 09 02:27:29.253939 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:29.253918 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-c69dff986-kcc28"] Sep 09 02:27:29.580718 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:29.580687 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2d68c24-03df-44b2-9bf1-5ef24b24577b" path="/var/lib/kubelet/pods/e2d68c24-03df-44b2-9bf1-5ef24b24577b/volumes" Sep 09 02:27:31.240392 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:31.240360 2577 generic.go:356] "Generic (PLEG): container finished" podID="3018c2e4-70dc-4421-8bb0-540126c40bd5" containerID="1fd62024aaa3065c1ccc8fba26ae89f558f911ae9cce7bbb221bd4b035157e76" exitCode=0 Sep 09 02:27:31.240784 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:31.240409 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" event={"ID":"3018c2e4-70dc-4421-8bb0-540126c40bd5","Type":"ContainerDied","Data":"1fd62024aaa3065c1ccc8fba26ae89f558f911ae9cce7bbb221bd4b035157e76"} Sep 09 02:27:31.240784 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:31.240726 2577 scope.go:117] "RemoveContainer" containerID="1fd62024aaa3065c1ccc8fba26ae89f558f911ae9cce7bbb221bd4b035157e76" Sep 09 02:27:31.572111 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:31.572076 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:31.631923 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:31.631891 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:31.687212 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:31.687176 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:32.246247 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:32.246213 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-ttjnc" event={"ID":"3018c2e4-70dc-4421-8bb0-540126c40bd5","Type":"ContainerStarted","Data":"fc265aad4358a1329fd917ce1b355a3632b1c0cfeaa1916bd3a40eba3db5cd81"} Sep 09 02:27:36.212692 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:36.212585 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:36.262934 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:36.262895 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:36.263560 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:36.263538 2577 generic.go:356] "Generic (PLEG): container finished" podID="15536224-40d3-43d7-b5df-c2a44121f29d" containerID="6519a76b42640c0001fd9d57f18ecaf8c2289e2b3399986c6369086d9ae2118e" exitCode=0 Sep 09 02:27:36.263657 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:36.263614 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" event={"ID":"15536224-40d3-43d7-b5df-c2a44121f29d","Type":"ContainerDied","Data":"6519a76b42640c0001fd9d57f18ecaf8c2289e2b3399986c6369086d9ae2118e"} Sep 09 02:27:36.263988 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:36.263975 2577 scope.go:117] "RemoveContainer" containerID="6519a76b42640c0001fd9d57f18ecaf8c2289e2b3399986c6369086d9ae2118e" Sep 09 02:27:37.269825 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:37.269789 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-lh5dp" event={"ID":"15536224-40d3-43d7-b5df-c2a44121f29d","Type":"ContainerStarted","Data":"84f462de9bcfc6616b37b13dc65d7b80b5842fe97617bbdf2af5d739f3926c92"} Sep 09 02:27:39.284784 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:39.284725 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:39.334517 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:39.334480 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:40.109842 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:40.109804 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:40.367682 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:40.367574 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:44.648102 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:44.648061 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:47.314005 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:47.313968 2577 generic.go:356] "Generic (PLEG): container finished" podID="1183493c-bda9-4396-858e-ee95549d8a22" containerID="5d68a8aec22f7ff1ef49596a217e6c94d9539deccb622c3f56a6164871561b60" exitCode=0 Sep 09 02:27:47.314397 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:47.314037 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" event={"ID":"1183493c-bda9-4396-858e-ee95549d8a22","Type":"ContainerDied","Data":"5d68a8aec22f7ff1ef49596a217e6c94d9539deccb622c3f56a6164871561b60"} Sep 09 02:27:47.314397 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:47.314339 2577 scope.go:117] "RemoveContainer" containerID="5d68a8aec22f7ff1ef49596a217e6c94d9539deccb622c3f56a6164871561b60" Sep 09 02:27:48.319855 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:48.319821 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-k87wb" event={"ID":"1183493c-bda9-4396-858e-ee95549d8a22","Type":"ContainerStarted","Data":"9ec20287ac9a72bb6000db4e693ac014a3028af4b05ae01f406caf2d073aa158"} Sep 09 02:27:48.704216 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:48.704136 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:48.808621 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:48.808591 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:59.749341 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:59.749307 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:59.803921 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:59.803885 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:27:59.856735 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:27:59.856703 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:28:17.141706 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:28:17.141669 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:28:55.373180 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:28:55.373101 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:28:55.427294 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:28:55.427251 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:01.470348 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:01.470302 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:01.526252 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:01.526209 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:01.616867 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:01.616837 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:01.673599 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:01.673570 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:02.242775 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:02.242738 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:02.297085 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:02.297052 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:07.189734 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:07.189700 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:07.291193 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:07.291158 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:07.343612 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:07.343567 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:12.044149 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:12.044109 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:12.143377 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:12.143337 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:12.195311 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:12.195265 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:14.831087 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:14.831052 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:14.881967 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:14.881936 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:21.227556 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:21.227518 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:21.326129 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:21.326091 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:43.552371 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:43.552326 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:29:43.648061 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:29:43.648028 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:08.778841 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:08.778774 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:08.834062 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:08.834030 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:10.161897 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:10.161863 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:10.950911 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:10.950853 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:13.785901 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:13.785863 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:13.839475 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:13.839445 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:15.164405 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:15.164375 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:15.267276 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:15.267242 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:15.372029 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:15.371996 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:15.465548 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:15.465458 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:17.556318 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:17.556289 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:17.773047 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:17.772997 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:20.378200 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:20.378163 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:20.478905 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:20.478871 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:21.935802 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:21.935768 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f"] Sep 09 02:30:21.936179 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:21.936050 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2d68c24-03df-44b2-9bf1-5ef24b24577b" containerName="registry" Sep 09 02:30:21.936179 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:21.936063 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2d68c24-03df-44b2-9bf1-5ef24b24577b" containerName="registry" Sep 09 02:30:21.936179 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:21.936122 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2d68c24-03df-44b2-9bf1-5ef24b24577b" containerName="registry" Sep 09 02:30:21.937807 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:21.937790 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f" Sep 09 02:30:21.940223 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:21.940200 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-d5d9q\"" Sep 09 02:30:21.940346 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:21.940331 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 09 02:30:21.941212 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:21.941193 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 09 02:30:21.945460 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:21.945441 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f"] Sep 09 02:30:22.018824 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:22.018786 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b149ec4-e302-41bd-a5d3-1db84ff110a6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-c5j5f\" (UID: \"2b149ec4-e302-41bd-a5d3-1db84ff110a6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f" Sep 09 02:30:22.018994 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:22.018857 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brth5\" (UniqueName: \"kubernetes.io/projected/2b149ec4-e302-41bd-a5d3-1db84ff110a6-kube-api-access-brth5\") pod \"cert-manager-cainjector-559cdd8c87-c5j5f\" (UID: \"2b149ec4-e302-41bd-a5d3-1db84ff110a6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f" Sep 09 02:30:22.119538 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:22.119502 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b149ec4-e302-41bd-a5d3-1db84ff110a6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-c5j5f\" (UID: \"2b149ec4-e302-41bd-a5d3-1db84ff110a6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f" Sep 09 02:30:22.119718 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:22.119573 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brth5\" (UniqueName: \"kubernetes.io/projected/2b149ec4-e302-41bd-a5d3-1db84ff110a6-kube-api-access-brth5\") pod \"cert-manager-cainjector-559cdd8c87-c5j5f\" (UID: \"2b149ec4-e302-41bd-a5d3-1db84ff110a6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f" Sep 09 02:30:22.126914 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:22.126885 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2b149ec4-e302-41bd-a5d3-1db84ff110a6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-c5j5f\" (UID: \"2b149ec4-e302-41bd-a5d3-1db84ff110a6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f" Sep 09 02:30:22.127087 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:22.127071 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brth5\" (UniqueName: \"kubernetes.io/projected/2b149ec4-e302-41bd-a5d3-1db84ff110a6-kube-api-access-brth5\") pod \"cert-manager-cainjector-559cdd8c87-c5j5f\" (UID: \"2b149ec4-e302-41bd-a5d3-1db84ff110a6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f" Sep 09 02:30:22.258371 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:22.258337 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f" Sep 09 02:30:22.527592 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:22.527483 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f"] Sep 09 02:30:22.530390 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:30:22.530358 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b149ec4_e302_41bd_a5d3_1db84ff110a6.slice/crio-2d4707b69a39652060f825a8adf7b76d326f28bcbbfb26ba0f954a001ffb18ce WatchSource:0}: Error finding container 2d4707b69a39652060f825a8adf7b76d326f28bcbbfb26ba0f954a001ffb18ce: Status 404 returned error can't find the container with id 2d4707b69a39652060f825a8adf7b76d326f28bcbbfb26ba0f954a001ffb18ce Sep 09 02:30:22.892570 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:22.892481 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f" event={"ID":"2b149ec4-e302-41bd-a5d3-1db84ff110a6","Type":"ContainerStarted","Data":"2d4707b69a39652060f825a8adf7b76d326f28bcbbfb26ba0f954a001ffb18ce"} Sep 09 02:30:24.770919 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:24.770884 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:25.907676 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:25.907640 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f" event={"ID":"2b149ec4-e302-41bd-a5d3-1db84ff110a6","Type":"ContainerStarted","Data":"66ee5115babb93d62b8df32c6eb49984137e52f0fd466fe90f319b2c8f11aad1"} Sep 09 02:30:25.926577 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:25.926522 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-c5j5f" podStartSLOduration=2.029432034 podStartE2EDuration="4.926505548s" totalImagesPullingTime="2.897073514s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:30:25.924615008 +0000 UTC m=+294.867183919" watchObservedRunningTime="2026-09-09 02:30:25.926505548 +0000 UTC m=+294.869074458" Sep 09 02:30:26.205547 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:26.205448 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:29.785340 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:29.785294 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:29.884970 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:29.884917 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:31.483995 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:31.483964 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/2.log" Sep 09 02:30:31.486081 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:31.486057 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/2.log" Sep 09 02:30:31.498987 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:31.498963 2577 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 02:30:36.036694 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:36.036613 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:37.194555 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:37.194520 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:37.710798 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:37.710759 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:53.580581 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:53.580532 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:53.699555 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:53.699511 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:54.387838 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.387801 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:54.620482 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.620451 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:54.720830 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.720750 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:54.751033 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.750997 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4"] Sep 09 02:30:54.754486 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.754463 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.756888 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.756863 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 09 02:30:54.761071 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.761048 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 09 02:30:54.761205 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.761056 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 09 02:30:54.761283 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.761114 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 09 02:30:54.761283 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.761148 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-w24vw\"" Sep 09 02:30:54.773646 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.773605 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4"] Sep 09 02:30:54.777418 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.777392 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-tmp\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.777541 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.777440 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-webhook-cert\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.777541 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.777508 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-manifests\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.777619 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.777566 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.777678 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.777615 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cthfv\" (UniqueName: \"kubernetes.io/projected/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-kube-api-access-cthfv\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.878774 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.878734 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cthfv\" (UniqueName: \"kubernetes.io/projected/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-kube-api-access-cthfv\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.878985 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.878784 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-tmp\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.878985 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.878810 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-webhook-cert\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.878985 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.878833 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-manifests\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.878985 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.878852 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.879379 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.879349 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-manifests\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.881323 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.881287 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-tmp\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.881526 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.881500 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-webhook-cert\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.881569 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.881555 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.886419 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.886387 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cthfv\" (UniqueName: \"kubernetes.io/projected/fe4c8400-b959-4fb0-ae5f-a6aed33fcad5-kube-api-access-cthfv\") pod \"opendatahub-operator-controller-manager-66fb449fdb-9btn4\" (UID: \"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5\") " pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:54.891328 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.891303 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:54.992609 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:54.992555 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:55.065125 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:55.065092 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:30:55.093560 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:55.093532 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:55.190214 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:55.190176 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:55.250871 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:55.250794 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:55.350460 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:55.350435 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4"] Sep 09 02:30:55.353102 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:30:55.353074 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4c8400_b959_4fb0_ae5f_a6aed33fcad5.slice/crio-379185a09050c29df1275834ac8b1afbdf82ddd526de080cff731a9cbf719081 WatchSource:0}: Error finding container 379185a09050c29df1275834ac8b1afbdf82ddd526de080cff731a9cbf719081: Status 404 returned error can't find the container with id 379185a09050c29df1275834ac8b1afbdf82ddd526de080cff731a9cbf719081 Sep 09 02:30:55.355029 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:55.355008 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 02:30:55.358826 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:55.358805 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:55.429407 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:55.429358 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:55.931152 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:55.931117 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:56.031513 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:56.031475 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" event={"ID":"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5","Type":"ContainerStarted","Data":"379185a09050c29df1275834ac8b1afbdf82ddd526de080cff731a9cbf719081"} Sep 09 02:30:56.049390 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:56.049355 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:30:59.052266 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:30:59.052231 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" event={"ID":"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5","Type":"ContainerStarted","Data":"c2af345f40bd4e5ef1d459c8503b605ce8f30c09daa303334222c5b85de5d5ed"} Sep 09 02:31:00.058389 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:00.058349 2577 generic.go:356] "Generic (PLEG): container finished" podID="fe4c8400-b959-4fb0-ae5f-a6aed33fcad5" containerID="c2af345f40bd4e5ef1d459c8503b605ce8f30c09daa303334222c5b85de5d5ed" exitCode=0 Sep 09 02:31:00.058785 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:00.058434 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" event={"ID":"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5","Type":"ContainerDied","Data":"c2af345f40bd4e5ef1d459c8503b605ce8f30c09daa303334222c5b85de5d5ed"} Sep 09 02:31:01.064338 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:01.064307 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" event={"ID":"fe4c8400-b959-4fb0-ae5f-a6aed33fcad5","Type":"ContainerStarted","Data":"2b7ed546a4013f61bccd162731c645803f0247f9a3986447b40c5cbb564baa86"} Sep 09 02:31:01.064739 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:01.064526 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:31:01.085275 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:01.085223 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" podStartSLOduration=2.196519495 podStartE2EDuration="7.085206384s" totalImagesPullingTime="3.850541779s" totalInitContainerRuntime="1.03814511s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:30:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:31:01.084059076 +0000 UTC m=+330.026627998" watchObservedRunningTime="2026-09-09 02:31:01.085206384 +0000 UTC m=+330.027775294" Sep 09 02:31:02.606624 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:02.606584 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:02.663595 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:02.663562 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:12.071675 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:12.071620 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66fb449fdb-9btn4" Sep 09 02:31:14.161419 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.161388 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:14.251301 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.251259 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:14.385815 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.385774 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:14.440240 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.440156 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-564645476c-8ds8p"] Sep 09 02:31:14.449658 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.449616 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" Sep 09 02:31:14.451929 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.451889 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 09 02:31:14.452186 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.452169 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-chfmm\"" Sep 09 02:31:14.452238 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.452207 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 09 02:31:14.452395 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.452373 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-564645476c-8ds8p"] Sep 09 02:31:14.542474 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.542439 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngkdv\" (UniqueName: \"kubernetes.io/projected/75f273ad-48b1-47a3-8d2a-c4e777c5786a-kube-api-access-ngkdv\") pod \"kube-auth-proxy-564645476c-8ds8p\" (UID: \"75f273ad-48b1-47a3-8d2a-c4e777c5786a\") " pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" Sep 09 02:31:14.542663 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.542510 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75f273ad-48b1-47a3-8d2a-c4e777c5786a-tmp\") pod \"kube-auth-proxy-564645476c-8ds8p\" (UID: \"75f273ad-48b1-47a3-8d2a-c4e777c5786a\") " pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" Sep 09 02:31:14.542663 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.542541 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75f273ad-48b1-47a3-8d2a-c4e777c5786a-tls-certs\") pod \"kube-auth-proxy-564645476c-8ds8p\" (UID: \"75f273ad-48b1-47a3-8d2a-c4e777c5786a\") " pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" Sep 09 02:31:14.559860 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.559829 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:14.643775 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.643741 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75f273ad-48b1-47a3-8d2a-c4e777c5786a-tmp\") pod \"kube-auth-proxy-564645476c-8ds8p\" (UID: \"75f273ad-48b1-47a3-8d2a-c4e777c5786a\") " pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" Sep 09 02:31:14.643977 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.643790 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75f273ad-48b1-47a3-8d2a-c4e777c5786a-tls-certs\") pod \"kube-auth-proxy-564645476c-8ds8p\" (UID: \"75f273ad-48b1-47a3-8d2a-c4e777c5786a\") " pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" Sep 09 02:31:14.643977 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.643848 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngkdv\" (UniqueName: \"kubernetes.io/projected/75f273ad-48b1-47a3-8d2a-c4e777c5786a-kube-api-access-ngkdv\") pod \"kube-auth-proxy-564645476c-8ds8p\" (UID: \"75f273ad-48b1-47a3-8d2a-c4e777c5786a\") " pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" Sep 09 02:31:14.646217 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.646190 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75f273ad-48b1-47a3-8d2a-c4e777c5786a-tmp\") pod \"kube-auth-proxy-564645476c-8ds8p\" (UID: \"75f273ad-48b1-47a3-8d2a-c4e777c5786a\") " pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" Sep 09 02:31:14.646415 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.646395 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75f273ad-48b1-47a3-8d2a-c4e777c5786a-tls-certs\") pod \"kube-auth-proxy-564645476c-8ds8p\" (UID: \"75f273ad-48b1-47a3-8d2a-c4e777c5786a\") " pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" Sep 09 02:31:14.651246 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.651220 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngkdv\" (UniqueName: \"kubernetes.io/projected/75f273ad-48b1-47a3-8d2a-c4e777c5786a-kube-api-access-ngkdv\") pod \"kube-auth-proxy-564645476c-8ds8p\" (UID: \"75f273ad-48b1-47a3-8d2a-c4e777c5786a\") " pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" Sep 09 02:31:14.767362 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:14.767324 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" Sep 09 02:31:15.037682 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.037588 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-564645476c-8ds8p"] Sep 09 02:31:15.040987 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:31:15.040957 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75f273ad_48b1_47a3_8d2a_c4e777c5786a.slice/crio-d0c3d49d2af2898db60cff9c2b4bdaef5480d8d8fdb5b3a8b7d3d3174c15fafb WatchSource:0}: Error finding container d0c3d49d2af2898db60cff9c2b4bdaef5480d8d8fdb5b3a8b7d3d3174c15fafb: Status 404 returned error can't find the container with id d0c3d49d2af2898db60cff9c2b4bdaef5480d8d8fdb5b3a8b7d3d3174c15fafb Sep 09 02:31:15.121615 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.121578 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" event={"ID":"75f273ad-48b1-47a3-8d2a-c4e777c5786a","Type":"ContainerStarted","Data":"d0c3d49d2af2898db60cff9c2b4bdaef5480d8d8fdb5b3a8b7d3d3174c15fafb"} Sep 09 02:31:15.710303 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.710267 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj"] Sep 09 02:31:15.713086 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.713055 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.715562 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.715427 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:15.715562 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.715518 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 09 02:31:15.715792 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.715611 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 09 02:31:15.715792 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.715678 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-w8jlw\"" Sep 09 02:31:15.716657 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.716554 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 09 02:31:15.716769 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.716673 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 09 02:31:15.728662 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.728584 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj"] Sep 09 02:31:15.781414 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.781379 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:15.855976 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.855937 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/30990b3d-e82b-4fc0-916e-8915ba9f28fa-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.856264 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.855985 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.856264 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.856025 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.856264 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.856095 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.856264 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.856137 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.856264 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.856185 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4kc2\" (UniqueName: \"kubernetes.io/projected/30990b3d-e82b-4fc0-916e-8915ba9f28fa-kube-api-access-p4kc2\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.856566 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.856282 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.956854 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.956811 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.957044 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.956875 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/30990b3d-e82b-4fc0-916e-8915ba9f28fa-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.957044 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.956906 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.957044 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.956940 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.957044 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.956974 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.957044 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.956999 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.957044 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.957031 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4kc2\" (UniqueName: \"kubernetes.io/projected/30990b3d-e82b-4fc0-916e-8915ba9f28fa-kube-api-access-p4kc2\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.958011 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.957974 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.960128 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.960083 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.960497 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.960433 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.960686 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.960667 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/30990b3d-e82b-4fc0-916e-8915ba9f28fa-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.961268 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.961227 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.970972 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.970946 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:15.971559 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:15.971539 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4kc2\" (UniqueName: \"kubernetes.io/projected/30990b3d-e82b-4fc0-916e-8915ba9f28fa-kube-api-access-p4kc2\") pod \"istiod-openshift-gateway-7b7995d5cd-cp5mj\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:16.025313 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:16.025277 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:16.312095 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:16.312021 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj"] Sep 09 02:31:16.314712 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:31:16.314675 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30990b3d_e82b_4fc0_916e_8915ba9f28fa.slice/crio-872607987fa65100d43c8357686aa835164ae3b3388c731b3129a496047582bb WatchSource:0}: Error finding container 872607987fa65100d43c8357686aa835164ae3b3388c731b3129a496047582bb: Status 404 returned error can't find the container with id 872607987fa65100d43c8357686aa835164ae3b3388c731b3129a496047582bb Sep 09 02:31:16.667565 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:16.667462 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:17.134329 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:17.134283 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" event={"ID":"30990b3d-e82b-4fc0-916e-8915ba9f28fa","Type":"ContainerStarted","Data":"872607987fa65100d43c8357686aa835164ae3b3388c731b3129a496047582bb"} Sep 09 02:31:18.143892 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:18.143857 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" event={"ID":"75f273ad-48b1-47a3-8d2a-c4e777c5786a","Type":"ContainerStarted","Data":"391da664ba46c7b9daceafe67c2874c9cd21ba0db416f62f5c3af1a7c4881936"} Sep 09 02:31:18.161534 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:18.161470 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-564645476c-8ds8p" podStartSLOduration=1.472425321 podStartE2EDuration="4.161415207s" totalImagesPullingTime="2.688989886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:31:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:31:18.160560164 +0000 UTC m=+347.103129075" watchObservedRunningTime="2026-09-09 02:31:18.161415207 +0000 UTC m=+347.103984119" Sep 09 02:31:22.614470 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:22.614428 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 02:31:22.614747 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:22.614502 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 02:31:23.166408 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:23.166366 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" event={"ID":"30990b3d-e82b-4fc0-916e-8915ba9f28fa","Type":"ContainerStarted","Data":"22b6b6465d81e1f4ddd10c08fb1232dada614ecac90cacc5303d82baa0f68bbe"} Sep 09 02:31:23.166612 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:23.166579 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:23.168253 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:23.168228 2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-cp5mj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 09 02:31:23.168343 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:23.168274 2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" podUID="30990b3d-e82b-4fc0-916e-8915ba9f28fa" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 02:31:23.185301 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:23.185257 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" podStartSLOduration=1.8883181530000002 podStartE2EDuration="8.185242509s" totalImagesPullingTime="6.296924356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:31:23.183175205 +0000 UTC m=+352.125744116" watchObservedRunningTime="2026-09-09 02:31:23.185242509 +0000 UTC m=+352.127811418" Sep 09 02:31:23.531005 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:23.530949 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:23.632928 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:23.632877 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:23.726272 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:23.726178 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:23.968591 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:23.968511 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:24.065088 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:24.065052 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:24.171190 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:24.171157 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:25.106501 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:25.106468 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:25.203100 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:25.203049 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:25.339663 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:25.339599 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:25.481592 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:25.481498 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:26.007531 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:26.007479 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:28.202581 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:28.202546 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:28.256792 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:28.256763 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:34.135104 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.135062 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:34.177641 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.177598 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd"] Sep 09 02:31:34.181351 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.181328 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" Sep 09 02:31:34.183966 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.183936 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 09 02:31:34.184096 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.183973 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-cvzhn\"" Sep 09 02:31:34.187298 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.187274 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q"] Sep 09 02:31:34.189466 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.189447 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" Sep 09 02:31:34.191707 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.191686 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd"] Sep 09 02:31:34.192120 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.192101 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 09 02:31:34.192206 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.192160 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-6mp99\"" Sep 09 02:31:34.203662 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.203623 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q"] Sep 09 02:31:34.223924 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.223857 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:34.295179 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.295146 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:34.309465 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.309434 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/402645c3-56ce-422a-a0d2-b27191896773-cert\") pod \"kserve-controller-manager-d6f4b65b8-w4fpd\" (UID: \"402645c3-56ce-422a-a0d2-b27191896773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" Sep 09 02:31:34.309614 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.309525 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/712d8722-7e34-43cd-b29d-18fe0638ab1c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vl89q\" (UID: \"712d8722-7e34-43cd-b29d-18fe0638ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" Sep 09 02:31:34.309614 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.309552 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfzjv\" (UniqueName: \"kubernetes.io/projected/712d8722-7e34-43cd-b29d-18fe0638ab1c-kube-api-access-sfzjv\") pod \"llmisvc-controller-manager-777c6bd7d5-vl89q\" (UID: \"712d8722-7e34-43cd-b29d-18fe0638ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" Sep 09 02:31:34.309614 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.309603 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdxr2\" (UniqueName: \"kubernetes.io/projected/402645c3-56ce-422a-a0d2-b27191896773-kube-api-access-rdxr2\") pod \"kserve-controller-manager-d6f4b65b8-w4fpd\" (UID: \"402645c3-56ce-422a-a0d2-b27191896773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" Sep 09 02:31:34.410410 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.410308 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/712d8722-7e34-43cd-b29d-18fe0638ab1c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vl89q\" (UID: \"712d8722-7e34-43cd-b29d-18fe0638ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" Sep 09 02:31:34.410410 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.410347 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfzjv\" (UniqueName: \"kubernetes.io/projected/712d8722-7e34-43cd-b29d-18fe0638ab1c-kube-api-access-sfzjv\") pod \"llmisvc-controller-manager-777c6bd7d5-vl89q\" (UID: \"712d8722-7e34-43cd-b29d-18fe0638ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" Sep 09 02:31:34.410410 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.410391 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdxr2\" (UniqueName: \"kubernetes.io/projected/402645c3-56ce-422a-a0d2-b27191896773-kube-api-access-rdxr2\") pod \"kserve-controller-manager-d6f4b65b8-w4fpd\" (UID: \"402645c3-56ce-422a-a0d2-b27191896773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" Sep 09 02:31:34.410689 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.410585 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/402645c3-56ce-422a-a0d2-b27191896773-cert\") pod \"kserve-controller-manager-d6f4b65b8-w4fpd\" (UID: \"402645c3-56ce-422a-a0d2-b27191896773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" Sep 09 02:31:34.410807 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:31:34.410784 2577 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Sep 09 02:31:34.410904 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:31:34.410891 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/402645c3-56ce-422a-a0d2-b27191896773-cert podName:402645c3-56ce-422a-a0d2-b27191896773 nodeName:}" failed. No retries permitted until 2026-09-09 02:31:34.910867133 +0000 UTC m=+363.853436035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/402645c3-56ce-422a-a0d2-b27191896773-cert") pod "kserve-controller-manager-d6f4b65b8-w4fpd" (UID: "402645c3-56ce-422a-a0d2-b27191896773") : secret "kserve-webhook-server-cert" not found Sep 09 02:31:34.413056 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.413033 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/712d8722-7e34-43cd-b29d-18fe0638ab1c-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vl89q\" (UID: \"712d8722-7e34-43cd-b29d-18fe0638ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" Sep 09 02:31:34.431701 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.431673 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfzjv\" (UniqueName: \"kubernetes.io/projected/712d8722-7e34-43cd-b29d-18fe0638ab1c-kube-api-access-sfzjv\") pod \"llmisvc-controller-manager-777c6bd7d5-vl89q\" (UID: \"712d8722-7e34-43cd-b29d-18fe0638ab1c\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" Sep 09 02:31:34.431819 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.431800 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdxr2\" (UniqueName: \"kubernetes.io/projected/402645c3-56ce-422a-a0d2-b27191896773-kube-api-access-rdxr2\") pod \"kserve-controller-manager-d6f4b65b8-w4fpd\" (UID: \"402645c3-56ce-422a-a0d2-b27191896773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" Sep 09 02:31:34.470003 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.469966 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:34.504811 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.504776 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" Sep 09 02:31:34.777409 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.777376 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q"] Sep 09 02:31:34.780765 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:31:34.780739 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod712d8722_7e34_43cd_b29d_18fe0638ab1c.slice/crio-20d6fa8cfeaa230d006327e81e322a09ddc472f2ba593805b1bb94f6daecd8bf WatchSource:0}: Error finding container 20d6fa8cfeaa230d006327e81e322a09ddc472f2ba593805b1bb94f6daecd8bf: Status 404 returned error can't find the container with id 20d6fa8cfeaa230d006327e81e322a09ddc472f2ba593805b1bb94f6daecd8bf Sep 09 02:31:34.796743 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.796720 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:34.916381 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.916322 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/402645c3-56ce-422a-a0d2-b27191896773-cert\") pod \"kserve-controller-manager-d6f4b65b8-w4fpd\" (UID: \"402645c3-56ce-422a-a0d2-b27191896773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" Sep 09 02:31:34.919202 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:34.919182 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/402645c3-56ce-422a-a0d2-b27191896773-cert\") pod \"kserve-controller-manager-d6f4b65b8-w4fpd\" (UID: \"402645c3-56ce-422a-a0d2-b27191896773\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" Sep 09 02:31:35.093721 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:35.093642 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" Sep 09 02:31:35.229808 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:35.229757 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" event={"ID":"712d8722-7e34-43cd-b29d-18fe0638ab1c","Type":"ContainerStarted","Data":"20d6fa8cfeaa230d006327e81e322a09ddc472f2ba593805b1bb94f6daecd8bf"} Sep 09 02:31:35.363503 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:35.363479 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd"] Sep 09 02:31:35.365915 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:31:35.365884 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod402645c3_56ce_422a_a0d2_b27191896773.slice/crio-e118ec87841ec82a27e830003be174a3afd34b9277fc5c94d9581d6ae9588ed8 WatchSource:0}: Error finding container e118ec87841ec82a27e830003be174a3afd34b9277fc5c94d9581d6ae9588ed8: Status 404 returned error can't find the container with id e118ec87841ec82a27e830003be174a3afd34b9277fc5c94d9581d6ae9588ed8 Sep 09 02:31:35.369459 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:35.369429 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:35.501912 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:35.501880 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:35.589744 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:35.589715 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:31:36.237599 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:36.237553 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" event={"ID":"402645c3-56ce-422a-a0d2-b27191896773","Type":"ContainerStarted","Data":"e118ec87841ec82a27e830003be174a3afd34b9277fc5c94d9581d6ae9588ed8"} Sep 09 02:31:36.600791 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:36.600751 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj"] Sep 09 02:31:36.601426 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:36.601014 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" podUID="30990b3d-e82b-4fc0-916e-8915ba9f28fa" containerName="discovery" containerID="cri-o://22b6b6465d81e1f4ddd10c08fb1232dada614ecac90cacc5303d82baa0f68bbe" gracePeriod=30 Sep 09 02:31:37.244986 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.244945 2577 generic.go:356] "Generic (PLEG): container finished" podID="30990b3d-e82b-4fc0-916e-8915ba9f28fa" containerID="22b6b6465d81e1f4ddd10c08fb1232dada614ecac90cacc5303d82baa0f68bbe" exitCode=0 Sep 09 02:31:37.245452 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.245021 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" event={"ID":"30990b3d-e82b-4fc0-916e-8915ba9f28fa","Type":"ContainerDied","Data":"22b6b6465d81e1f4ddd10c08fb1232dada614ecac90cacc5303d82baa0f68bbe"} Sep 09 02:31:37.424719 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.424691 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:37.444649 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.443973 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4kc2\" (UniqueName: \"kubernetes.io/projected/30990b3d-e82b-4fc0-916e-8915ba9f28fa-kube-api-access-p4kc2\") pod \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " Sep 09 02:31:37.444649 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.444024 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-kubeconfig\") pod \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " Sep 09 02:31:37.444649 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.444055 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-csr-dns-cert\") pod \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " Sep 09 02:31:37.444649 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.444086 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-token\") pod \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " Sep 09 02:31:37.444649 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.444120 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/30990b3d-e82b-4fc0-916e-8915ba9f28fa-local-certs\") pod \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " Sep 09 02:31:37.444649 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.444148 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-cacerts\") pod \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " Sep 09 02:31:37.444649 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.444235 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-csr-ca-configmap\") pod \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\" (UID: \"30990b3d-e82b-4fc0-916e-8915ba9f28fa\") " Sep 09 02:31:37.445362 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.445321 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "30990b3d-e82b-4fc0-916e-8915ba9f28fa" (UID: "30990b3d-e82b-4fc0-916e-8915ba9f28fa"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:31:37.448482 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.448409 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-cacerts" (OuterVolumeSpecName: "cacerts") pod "30990b3d-e82b-4fc0-916e-8915ba9f28fa" (UID: "30990b3d-e82b-4fc0-916e-8915ba9f28fa"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:31:37.448578 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.448488 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-token" (OuterVolumeSpecName: "istio-token") pod "30990b3d-e82b-4fc0-916e-8915ba9f28fa" (UID: "30990b3d-e82b-4fc0-916e-8915ba9f28fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:31:37.449014 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.448972 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30990b3d-e82b-4fc0-916e-8915ba9f28fa-kube-api-access-p4kc2" (OuterVolumeSpecName: "kube-api-access-p4kc2") pod "30990b3d-e82b-4fc0-916e-8915ba9f28fa" (UID: "30990b3d-e82b-4fc0-916e-8915ba9f28fa"). InnerVolumeSpecName "kube-api-access-p4kc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:31:37.449171 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.449059 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "30990b3d-e82b-4fc0-916e-8915ba9f28fa" (UID: "30990b3d-e82b-4fc0-916e-8915ba9f28fa"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:31:37.449535 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.449207 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "30990b3d-e82b-4fc0-916e-8915ba9f28fa" (UID: "30990b3d-e82b-4fc0-916e-8915ba9f28fa"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:31:37.450745 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.450695 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30990b3d-e82b-4fc0-916e-8915ba9f28fa-local-certs" (OuterVolumeSpecName: "local-certs") pod "30990b3d-e82b-4fc0-916e-8915ba9f28fa" (UID: "30990b3d-e82b-4fc0-916e-8915ba9f28fa"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 02:31:37.545157 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.545069 2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-csr-ca-configmap\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:31:37.545157 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.545102 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4kc2\" (UniqueName: \"kubernetes.io/projected/30990b3d-e82b-4fc0-916e-8915ba9f28fa-kube-api-access-p4kc2\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:31:37.545157 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.545113 2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-kubeconfig\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:31:37.545157 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.545121 2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-csr-dns-cert\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:31:37.545157 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.545130 2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30990b3d-e82b-4fc0-916e-8915ba9f28fa-istio-token\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:31:37.545157 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.545139 2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/30990b3d-e82b-4fc0-916e-8915ba9f28fa-local-certs\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:31:37.545157 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:37.545153 2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/30990b3d-e82b-4fc0-916e-8915ba9f28fa-cacerts\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:31:38.250555 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:38.250469 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" Sep 09 02:31:38.250555 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:38.250487 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj" event={"ID":"30990b3d-e82b-4fc0-916e-8915ba9f28fa","Type":"ContainerDied","Data":"872607987fa65100d43c8357686aa835164ae3b3388c731b3129a496047582bb"} Sep 09 02:31:38.250555 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:38.250545 2577 scope.go:117] "RemoveContainer" containerID="22b6b6465d81e1f4ddd10c08fb1232dada614ecac90cacc5303d82baa0f68bbe" Sep 09 02:31:38.252216 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:38.252183 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" event={"ID":"402645c3-56ce-422a-a0d2-b27191896773","Type":"ContainerStarted","Data":"9e047eced33305b86df8ffb89c70b047494a4ee70bcd74cb6133dda519951284"} Sep 09 02:31:38.252320 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:38.252261 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" Sep 09 02:31:38.253958 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:38.253935 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" event={"ID":"712d8722-7e34-43cd-b29d-18fe0638ab1c","Type":"ContainerStarted","Data":"c4b12ad8336d4c105d82292f89d9ca6ca0d937977d502b2a3b551c9a7cc631e4"} Sep 09 02:31:38.254045 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:38.254009 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" Sep 09 02:31:38.276677 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:38.276608 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" podStartSLOduration=1.6453624420000001 podStartE2EDuration="4.276591122s" totalImagesPullingTime="2.63122868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:31:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:31:38.275041334 +0000 UTC m=+367.217610248" watchObservedRunningTime="2026-09-09 02:31:38.276591122 +0000 UTC m=+367.219160048" Sep 09 02:31:38.288624 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:38.288591 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj"] Sep 09 02:31:38.294118 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:38.294086 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cp5mj"] Sep 09 02:31:38.307575 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:38.307521 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" podStartSLOduration=1.092452192 podStartE2EDuration="4.307504664s" totalImagesPullingTime="3.215052472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:31:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:31:38.306430553 +0000 UTC m=+367.248999457" watchObservedRunningTime="2026-09-09 02:31:38.307504664 +0000 UTC m=+367.250073574" Sep 09 02:31:39.581449 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:31:39.581404 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30990b3d-e82b-4fc0-916e-8915ba9f28fa" path="/var/lib/kubelet/pods/30990b3d-e82b-4fc0-916e-8915ba9f28fa/volumes" Sep 09 02:32:09.272624 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:09.272588 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vl89q" Sep 09 02:32:09.273157 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:09.272688 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-w4fpd" Sep 09 02:32:17.070999 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:17.070960 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:17.133641 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:17.133601 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:17.209619 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:17.209586 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:17.310591 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:17.310561 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:17.804547 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:17.804516 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:17.872483 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:17.872433 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:26.375363 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:26.375318 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:26.474536 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:26.474502 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:27.902494 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:27.902456 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:28.000644 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:28.000603 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:28.101219 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:28.101179 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:39.599133 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:39.599102 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:41.643460 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.643415 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg"] Sep 09 02:32:41.643936 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.643840 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30990b3d-e82b-4fc0-916e-8915ba9f28fa" containerName="discovery" Sep 09 02:32:41.643936 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.643857 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="30990b3d-e82b-4fc0-916e-8915ba9f28fa" containerName="discovery" Sep 09 02:32:41.644023 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.643944 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="30990b3d-e82b-4fc0-916e-8915ba9f28fa" containerName="discovery" Sep 09 02:32:41.648213 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.648189 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg" Sep 09 02:32:41.650668 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.650624 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 09 02:32:41.650798 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.650737 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-sj7wb\"" Sep 09 02:32:41.650798 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.650791 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 09 02:32:41.650922 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.650793 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 09 02:32:41.654694 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.654668 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg"] Sep 09 02:32:41.773014 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.772976 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kl27\" (UniqueName: \"kubernetes.io/projected/0c56197e-99e8-4392-bf0d-cfc0e36555e6-kube-api-access-5kl27\") pod \"dns-operator-controller-manager-7b84cd7c9c-zg7pg\" (UID: \"0c56197e-99e8-4392-bf0d-cfc0e36555e6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg" Sep 09 02:32:41.874303 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.874263 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kl27\" (UniqueName: \"kubernetes.io/projected/0c56197e-99e8-4392-bf0d-cfc0e36555e6-kube-api-access-5kl27\") pod \"dns-operator-controller-manager-7b84cd7c9c-zg7pg\" (UID: \"0c56197e-99e8-4392-bf0d-cfc0e36555e6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg" Sep 09 02:32:41.887266 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.887241 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kl27\" (UniqueName: \"kubernetes.io/projected/0c56197e-99e8-4392-bf0d-cfc0e36555e6-kube-api-access-5kl27\") pod \"dns-operator-controller-manager-7b84cd7c9c-zg7pg\" (UID: \"0c56197e-99e8-4392-bf0d-cfc0e36555e6\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg" Sep 09 02:32:41.959129 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:41.959041 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg" Sep 09 02:32:42.205778 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:42.205742 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:42.229812 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:42.229787 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg"] Sep 09 02:32:42.232437 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:32:42.232397 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c56197e_99e8_4392_bf0d_cfc0e36555e6.slice/crio-2d8662ba34fe57fc92926f633d18259eabc6af99367db9bee57aff508f3cba8d WatchSource:0}: Error finding container 2d8662ba34fe57fc92926f633d18259eabc6af99367db9bee57aff508f3cba8d: Status 404 returned error can't find the container with id 2d8662ba34fe57fc92926f633d18259eabc6af99367db9bee57aff508f3cba8d Sep 09 02:32:42.537714 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:42.537670 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg" event={"ID":"0c56197e-99e8-4392-bf0d-cfc0e36555e6","Type":"ContainerStarted","Data":"2d8662ba34fe57fc92926f633d18259eabc6af99367db9bee57aff508f3cba8d"} Sep 09 02:32:42.655199 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:42.655165 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:43.002602 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:43.002513 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:44.031048 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:44.031011 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:45.556595 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:45.556553 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg" event={"ID":"0c56197e-99e8-4392-bf0d-cfc0e36555e6","Type":"ContainerStarted","Data":"6d09a559b5ef72df109b4a410dc6fa131e7e3ec03e2af9d476bdc16bc057c6eb"} Sep 09 02:32:45.557010 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:45.556701 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg" Sep 09 02:32:45.574518 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:45.574465 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg" podStartSLOduration=2.054587672 podStartE2EDuration="4.574449839s" totalImagesPullingTime="2.519862167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:32:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:32:45.573075151 +0000 UTC m=+434.515644061" watchObservedRunningTime="2026-09-09 02:32:45.574449839 +0000 UTC m=+434.517018750" Sep 09 02:32:45.843470 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:45.843388 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v"] Sep 09 02:32:45.846733 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:45.846716 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:32:45.849710 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:45.849686 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-z5lbb\"" Sep 09 02:32:45.866804 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:45.866777 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v"] Sep 09 02:32:45.895197 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:45.895165 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:45.908370 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:45.908341 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qzxc\" (UniqueName: \"kubernetes.io/projected/4f4abbc8-223d-4381-b5d8-cdcf22ca890a-kube-api-access-5qzxc\") pod \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" (UID: \"4f4abbc8-223d-4381-b5d8-cdcf22ca890a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:32:45.908452 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:45.908411 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f4abbc8-223d-4381-b5d8-cdcf22ca890a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" (UID: \"4f4abbc8-223d-4381-b5d8-cdcf22ca890a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:32:46.008571 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:46.008524 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:46.009048 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:46.009018 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f4abbc8-223d-4381-b5d8-cdcf22ca890a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" (UID: \"4f4abbc8-223d-4381-b5d8-cdcf22ca890a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:32:46.009193 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:46.009109 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qzxc\" (UniqueName: \"kubernetes.io/projected/4f4abbc8-223d-4381-b5d8-cdcf22ca890a-kube-api-access-5qzxc\") pod \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" (UID: \"4f4abbc8-223d-4381-b5d8-cdcf22ca890a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:32:46.009489 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:46.009466 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f4abbc8-223d-4381-b5d8-cdcf22ca890a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" (UID: \"4f4abbc8-223d-4381-b5d8-cdcf22ca890a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:32:46.025136 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:46.025102 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qzxc\" (UniqueName: \"kubernetes.io/projected/4f4abbc8-223d-4381-b5d8-cdcf22ca890a-kube-api-access-5qzxc\") pod \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" (UID: \"4f4abbc8-223d-4381-b5d8-cdcf22ca890a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:32:46.156574 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:46.156475 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:32:46.442142 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:46.442030 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:46.442857 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:46.442833 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v"] Sep 09 02:32:46.445777 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:32:46.445744 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f4abbc8_223d_4381_b5d8_cdcf22ca890a.slice/crio-268aad5999a734f366c1dfc6b05fda0c5f5a97c6a3806c9db4005e4aedc4c7f5 WatchSource:0}: Error finding container 268aad5999a734f366c1dfc6b05fda0c5f5a97c6a3806c9db4005e4aedc4c7f5: Status 404 returned error can't find the container with id 268aad5999a734f366c1dfc6b05fda0c5f5a97c6a3806c9db4005e4aedc4c7f5 Sep 09 02:32:46.565705 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:46.565658 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" event={"ID":"4f4abbc8-223d-4381-b5d8-cdcf22ca890a","Type":"ContainerStarted","Data":"268aad5999a734f366c1dfc6b05fda0c5f5a97c6a3806c9db4005e4aedc4c7f5"} Sep 09 02:32:47.447561 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:47.447528 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:48.405194 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:48.405156 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-swwcp"] Sep 09 02:32:48.408801 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:48.408774 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-swwcp" Sep 09 02:32:48.411126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:48.411106 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-k2jmg\"" Sep 09 02:32:48.421332 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:48.421303 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-swwcp"] Sep 09 02:32:48.425499 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:48.425470 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bw42\" (UniqueName: \"kubernetes.io/projected/c22d3db0-d742-4316-87ab-0de3d06e1c00-kube-api-access-8bw42\") pod \"authorino-operator-76b478bd78-swwcp\" (UID: \"c22d3db0-d742-4316-87ab-0de3d06e1c00\") " pod="kuadrant-system/authorino-operator-76b478bd78-swwcp" Sep 09 02:32:48.526912 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:48.526866 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bw42\" (UniqueName: \"kubernetes.io/projected/c22d3db0-d742-4316-87ab-0de3d06e1c00-kube-api-access-8bw42\") pod \"authorino-operator-76b478bd78-swwcp\" (UID: \"c22d3db0-d742-4316-87ab-0de3d06e1c00\") " pod="kuadrant-system/authorino-operator-76b478bd78-swwcp" Sep 09 02:32:48.535399 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:48.535361 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bw42\" (UniqueName: \"kubernetes.io/projected/c22d3db0-d742-4316-87ab-0de3d06e1c00-kube-api-access-8bw42\") pod \"authorino-operator-76b478bd78-swwcp\" (UID: \"c22d3db0-d742-4316-87ab-0de3d06e1c00\") " pod="kuadrant-system/authorino-operator-76b478bd78-swwcp" Sep 09 02:32:48.721131 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:48.721039 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-swwcp" Sep 09 02:32:49.005302 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:49.005278 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:49.005741 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:49.005713 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-swwcp"] Sep 09 02:32:49.008371 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:32:49.008343 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22d3db0_d742_4316_87ab_0de3d06e1c00.slice/crio-d739edd52955b8d253b8519085d59e39acf1aae6faeda5ee54aa0bf7ab036a28 WatchSource:0}: Error finding container d739edd52955b8d253b8519085d59e39acf1aae6faeda5ee54aa0bf7ab036a28: Status 404 returned error can't find the container with id d739edd52955b8d253b8519085d59e39acf1aae6faeda5ee54aa0bf7ab036a28 Sep 09 02:32:49.243040 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:49.242989 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:49.581613 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:49.581579 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-swwcp" event={"ID":"c22d3db0-d742-4316-87ab-0de3d06e1c00","Type":"ContainerStarted","Data":"d739edd52955b8d253b8519085d59e39acf1aae6faeda5ee54aa0bf7ab036a28"} Sep 09 02:32:50.604195 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:50.604160 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75"] Sep 09 02:32:50.607978 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:50.607955 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:32:50.610359 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:50.610338 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-fhr88\"" Sep 09 02:32:50.616944 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:50.616916 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75"] Sep 09 02:32:50.642920 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:50.642886 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsmnn\" (UniqueName: \"kubernetes.io/projected/b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd-kube-api-access-lsmnn\") pod \"limitador-operator-controller-manager-74495c69f8-7nf75\" (UID: \"b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:32:50.743314 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:50.743269 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsmnn\" (UniqueName: \"kubernetes.io/projected/b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd-kube-api-access-lsmnn\") pod \"limitador-operator-controller-manager-74495c69f8-7nf75\" (UID: \"b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:32:50.751494 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:50.751468 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsmnn\" (UniqueName: \"kubernetes.io/projected/b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd-kube-api-access-lsmnn\") pod \"limitador-operator-controller-manager-74495c69f8-7nf75\" (UID: \"b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:32:50.920008 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:50.919924 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:32:51.269001 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:51.268973 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75"] Sep 09 02:32:51.271555 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:32:51.271531 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7135472_7ebf_4cc8_b91f_eb4fc9ddc0bd.slice/crio-d36b7ee1b43991475fa4a9afeb51cc454693cded3cc204da2c65d802a32c9be8 WatchSource:0}: Error finding container d36b7ee1b43991475fa4a9afeb51cc454693cded3cc204da2c65d802a32c9be8: Status 404 returned error can't find the container with id d36b7ee1b43991475fa4a9afeb51cc454693cded3cc204da2c65d802a32c9be8 Sep 09 02:32:51.591597 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:51.591505 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-swwcp" event={"ID":"c22d3db0-d742-4316-87ab-0de3d06e1c00","Type":"ContainerStarted","Data":"f1be66a736e9dd5255b6fad8fa9d734a4de6891e271ab142842e58fdba85a92a"} Sep 09 02:32:51.591765 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:51.591653 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-swwcp" Sep 09 02:32:51.592844 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:51.592815 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" event={"ID":"b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd","Type":"ContainerStarted","Data":"d36b7ee1b43991475fa4a9afeb51cc454693cded3cc204da2c65d802a32c9be8"} Sep 09 02:32:51.610329 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:51.610282 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-swwcp" podStartSLOduration=1.54520525 podStartE2EDuration="3.610266276s" totalImagesPullingTime="2.065061026s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:32:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:32:51.609254159 +0000 UTC m=+440.551823081" watchObservedRunningTime="2026-09-09 02:32:51.610266276 +0000 UTC m=+440.552835181" Sep 09 02:32:51.998723 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:51.998695 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:52.096907 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:52.096856 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:54.611120 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:54.611082 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" event={"ID":"4f4abbc8-223d-4381-b5d8-cdcf22ca890a","Type":"ContainerStarted","Data":"98cdf30c23ea6c418f5bf9e727f2537b93f0036b799ad18c809fc7e8d3a2404e"} Sep 09 02:32:54.611514 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:54.611168 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:32:54.632918 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:54.632863 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" podStartSLOduration=2.229167045 podStartE2EDuration="9.632846959s" totalImagesPullingTime="7.403679914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:32:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:32:54.630272047 +0000 UTC m=+443.572840956" watchObservedRunningTime="2026-09-09 02:32:54.632846959 +0000 UTC m=+443.575415868" Sep 09 02:32:56.568262 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.568226 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-zg7pg" Sep 09 02:32:56.623561 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.623511 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" event={"ID":"b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd","Type":"ContainerStarted","Data":"65e27342454515f67ac88e9653391323d8a7390337d8cc7fbe8c2589c570e598"} Sep 09 02:32:56.623762 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.623623 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:32:56.642374 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.642276 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" podStartSLOduration=1.708746136 podStartE2EDuration="6.642259366s" totalImagesPullingTime="4.93351323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:32:56.641465351 +0000 UTC m=+445.584034261" watchObservedRunningTime="2026-09-09 02:32:56.642259366 +0000 UTC m=+445.584828276" Sep 09 02:32:56.781153 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.781117 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45"] Sep 09 02:32:56.784794 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.784772 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" Sep 09 02:32:56.787121 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.787095 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:56.787250 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.787206 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Sep 09 02:32:56.787349 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.787329 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9v59b\"" Sep 09 02:32:56.787414 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.787397 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Sep 09 02:32:56.789757 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.789733 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f46f2b07-7dce-4923-9d4c-6a2f4052640c-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-j2c45\" (UID: \"f46f2b07-7dce-4923-9d4c-6a2f4052640c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" Sep 09 02:32:56.789883 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.789813 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn7t6\" (UniqueName: \"kubernetes.io/projected/f46f2b07-7dce-4923-9d4c-6a2f4052640c-kube-api-access-fn7t6\") pod \"kuadrant-console-plugin-fbdd4f445-j2c45\" (UID: \"f46f2b07-7dce-4923-9d4c-6a2f4052640c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" Sep 09 02:32:56.789883 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.789844 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f46f2b07-7dce-4923-9d4c-6a2f4052640c-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-j2c45\" (UID: \"f46f2b07-7dce-4923-9d4c-6a2f4052640c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" Sep 09 02:32:56.791420 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.791396 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45"] Sep 09 02:32:56.891085 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.890988 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f46f2b07-7dce-4923-9d4c-6a2f4052640c-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-j2c45\" (UID: \"f46f2b07-7dce-4923-9d4c-6a2f4052640c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" Sep 09 02:32:56.891085 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.891036 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn7t6\" (UniqueName: \"kubernetes.io/projected/f46f2b07-7dce-4923-9d4c-6a2f4052640c-kube-api-access-fn7t6\") pod \"kuadrant-console-plugin-fbdd4f445-j2c45\" (UID: \"f46f2b07-7dce-4923-9d4c-6a2f4052640c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" Sep 09 02:32:56.891085 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.891070 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f46f2b07-7dce-4923-9d4c-6a2f4052640c-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-j2c45\" (UID: \"f46f2b07-7dce-4923-9d4c-6a2f4052640c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" Sep 09 02:32:56.891884 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.891854 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f46f2b07-7dce-4923-9d4c-6a2f4052640c-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-j2c45\" (UID: \"f46f2b07-7dce-4923-9d4c-6a2f4052640c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" Sep 09 02:32:56.893738 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.893718 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f46f2b07-7dce-4923-9d4c-6a2f4052640c-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-j2c45\" (UID: \"f46f2b07-7dce-4923-9d4c-6a2f4052640c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" Sep 09 02:32:56.905763 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.905733 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn7t6\" (UniqueName: \"kubernetes.io/projected/f46f2b07-7dce-4923-9d4c-6a2f4052640c-kube-api-access-fn7t6\") pod \"kuadrant-console-plugin-fbdd4f445-j2c45\" (UID: \"f46f2b07-7dce-4923-9d4c-6a2f4052640c\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" Sep 09 02:32:56.940689 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:56.940657 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:57.095686 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:57.095651 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" Sep 09 02:32:57.364247 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:57.364220 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45"] Sep 09 02:32:57.366333 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:32:57.366303 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf46f2b07_7dce_4923_9d4c_6a2f4052640c.slice/crio-f1c0425eb322dcbba14e12c9640918b31844722684614c9706d69ce59d808314 WatchSource:0}: Error finding container f1c0425eb322dcbba14e12c9640918b31844722684614c9706d69ce59d808314: Status 404 returned error can't find the container with id f1c0425eb322dcbba14e12c9640918b31844722684614c9706d69ce59d808314 Sep 09 02:32:57.412305 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:57.412230 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:57.518834 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:57.518793 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:57.612755 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:57.612720 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:57.630097 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:57.630061 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" event={"ID":"f46f2b07-7dce-4923-9d4c-6a2f4052640c","Type":"ContainerStarted","Data":"f1c0425eb322dcbba14e12c9640918b31844722684614c9706d69ce59d808314"} Sep 09 02:32:57.671244 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:57.671165 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:57.803141 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:57.803113 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:32:57.856116 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:32:57.856086 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:01.221816 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:01.221777 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:01.319802 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:01.319768 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:02.601736 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:02.601706 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-swwcp" Sep 09 02:33:03.665063 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:03.665030 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" event={"ID":"f46f2b07-7dce-4923-9d4c-6a2f4052640c","Type":"ContainerStarted","Data":"87aa95b4d0815b249ea06608c56bb1479c82098357632b765f532010f70d2c9c"} Sep 09 02:33:03.683727 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:03.683678 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-j2c45" podStartSLOduration=2.075036952 podStartE2EDuration="7.683662842s" totalImagesPullingTime="5.60862589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:32:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:33:03.681326926 +0000 UTC m=+452.623895835" watchObservedRunningTime="2026-09-09 02:33:03.683662842 +0000 UTC m=+452.626231752" Sep 09 02:33:05.397369 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:05.397336 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:05.497272 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:05.497239 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:05.618019 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:05.617989 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:06.814040 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:06.814006 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57"] Sep 09 02:33:06.817599 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:06.817581 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:06.829367 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:06.829339 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57"] Sep 09 02:33:06.875370 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:06.875335 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da40869d-0781-4a15-b26b-a7163daa8eb1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-7cx57\" (UID: \"da40869d-0781-4a15-b26b-a7163daa8eb1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:06.875544 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:06.875394 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw7vm\" (UniqueName: \"kubernetes.io/projected/da40869d-0781-4a15-b26b-a7163daa8eb1-kube-api-access-rw7vm\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-7cx57\" (UID: \"da40869d-0781-4a15-b26b-a7163daa8eb1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:06.975951 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:06.975909 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw7vm\" (UniqueName: \"kubernetes.io/projected/da40869d-0781-4a15-b26b-a7163daa8eb1-kube-api-access-rw7vm\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-7cx57\" (UID: \"da40869d-0781-4a15-b26b-a7163daa8eb1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:06.976116 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:06.976001 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da40869d-0781-4a15-b26b-a7163daa8eb1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-7cx57\" (UID: \"da40869d-0781-4a15-b26b-a7163daa8eb1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:06.976382 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:06.976359 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da40869d-0781-4a15-b26b-a7163daa8eb1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-7cx57\" (UID: \"da40869d-0781-4a15-b26b-a7163daa8eb1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:06.984059 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:06.984030 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw7vm\" (UniqueName: \"kubernetes.io/projected/da40869d-0781-4a15-b26b-a7163daa8eb1-kube-api-access-rw7vm\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-7cx57\" (UID: \"da40869d-0781-4a15-b26b-a7163daa8eb1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:07.127418 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:07.127329 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:07.404599 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:07.404572 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57"] Sep 09 02:33:07.405658 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:33:07.405613 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda40869d_0781_4a15_b26b_a7163daa8eb1.slice/crio-5d4feda862a22e2922eaa267099d7843c7c9a0c261ada12ad82d8406c9a57948 WatchSource:0}: Error finding container 5d4feda862a22e2922eaa267099d7843c7c9a0c261ada12ad82d8406c9a57948: Status 404 returned error can't find the container with id 5d4feda862a22e2922eaa267099d7843c7c9a0c261ada12ad82d8406c9a57948 Sep 09 02:33:07.632464 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:07.632429 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:33:07.688328 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:07.688236 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" event={"ID":"da40869d-0781-4a15-b26b-a7163daa8eb1","Type":"ContainerStarted","Data":"b68226cd8169b7306755e54fb24d1eb5cbf25bd19799154ce267b937ffc2c474"} Sep 09 02:33:07.688328 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:07.688300 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" event={"ID":"da40869d-0781-4a15-b26b-a7163daa8eb1","Type":"ContainerStarted","Data":"5d4feda862a22e2922eaa267099d7843c7c9a0c261ada12ad82d8406c9a57948"} Sep 09 02:33:07.688565 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:07.688340 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:07.707997 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:07.707945 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" podStartSLOduration=1.7079278 podStartE2EDuration="1.7079278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:33:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:33:07.706530629 +0000 UTC m=+456.649099551" watchObservedRunningTime="2026-09-09 02:33:07.7079278 +0000 UTC m=+456.650496710" Sep 09 02:33:08.045080 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.045046 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57"] Sep 09 02:33:08.055590 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.055555 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57"] Sep 09 02:33:08.067705 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.067667 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v"] Sep 09 02:33:08.067896 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.067873 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" containerName="manager" containerID="cri-o://98cdf30c23ea6c418f5bf9e727f2537b93f0036b799ad18c809fc7e8d3a2404e" gracePeriod=2 Sep 09 02:33:08.074663 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.074603 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v"] Sep 09 02:33:08.079898 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.079858 2577 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"4f4abbc8-223d-4381-b5d8-cdcf22ca890a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-drn6v\": pods \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:08.081058 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.080962 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc"] Sep 09 02:33:08.081445 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.081423 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da40869d-0781-4a15-b26b-a7163daa8eb1" containerName="manager" Sep 09 02:33:08.081554 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.081447 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="da40869d-0781-4a15-b26b-a7163daa8eb1" containerName="manager" Sep 09 02:33:08.081554 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.081476 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" containerName="manager" Sep 09 02:33:08.081554 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.081485 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" containerName="manager" Sep 09 02:33:08.081742 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.081581 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" containerName="manager" Sep 09 02:33:08.081742 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.081592 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="da40869d-0781-4a15-b26b-a7163daa8eb1" containerName="manager" Sep 09 02:33:08.092126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.092008 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" Sep 09 02:33:08.093101 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.092556 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75"] Sep 09 02:33:08.093101 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.092749 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" containerName="manager" containerID="cri-o://65e27342454515f67ac88e9653391323d8a7390337d8cc7fbe8c2589c570e598" gracePeriod=2 Sep 09 02:33:08.095039 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.095008 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:08.101667 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.101518 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75"] Sep 09 02:33:08.104118 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.104086 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc"] Sep 09 02:33:08.108621 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.108574 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq"] Sep 09 02:33:08.109125 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.109104 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" containerName="manager" Sep 09 02:33:08.109174 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.109130 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" containerName="manager" Sep 09 02:33:08.109243 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.109231 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" containerName="manager" Sep 09 02:33:08.112548 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.112530 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" Sep 09 02:33:08.124728 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.124702 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh"] Sep 09 02:33:08.128478 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.128453 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh" Sep 09 02:33:08.141182 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.141114 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq"] Sep 09 02:33:08.142989 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.142965 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh"] Sep 09 02:33:08.166394 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.166356 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:08.168564 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.168527 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7nf75\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:33:08.188075 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.188044 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh9bx\" (UniqueName: \"kubernetes.io/projected/a0cbe25b-3f1b-4c95-85ab-a05da38d1490-kube-api-access-nh9bx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b9skq\" (UID: \"a0cbe25b-3f1b-4c95-85ab-a05da38d1490\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" Sep 09 02:33:08.188075 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.188085 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aba1f79-1ca8-483e-921c-fd1ab682d9e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-gjknc\" (UID: \"7aba1f79-1ca8-483e-921c-fd1ab682d9e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" Sep 09 02:33:08.188275 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.188166 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlgcr\" (UniqueName: \"kubernetes.io/projected/7aba1f79-1ca8-483e-921c-fd1ab682d9e5-kube-api-access-jlgcr\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-gjknc\" (UID: \"7aba1f79-1ca8-483e-921c-fd1ab682d9e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" Sep 09 02:33:08.188275 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.188224 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0cbe25b-3f1b-4c95-85ab-a05da38d1490-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b9skq\" (UID: \"a0cbe25b-3f1b-4c95-85ab-a05da38d1490\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" Sep 09 02:33:08.188275 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.188269 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv9tb\" (UniqueName: \"kubernetes.io/projected/7afcb052-d8ca-4a7e-9d85-0fcce7270162-kube-api-access-fv9tb\") pod \"limitador-operator-controller-manager-74495c69f8-r69dh\" (UID: \"7afcb052-d8ca-4a7e-9d85-0fcce7270162\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh" Sep 09 02:33:08.194130 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.194098 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:08.196195 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.196167 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7nf75\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:33:08.289308 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.289270 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh9bx\" (UniqueName: \"kubernetes.io/projected/a0cbe25b-3f1b-4c95-85ab-a05da38d1490-kube-api-access-nh9bx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b9skq\" (UID: \"a0cbe25b-3f1b-4c95-85ab-a05da38d1490\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" Sep 09 02:33:08.289308 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.289309 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aba1f79-1ca8-483e-921c-fd1ab682d9e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-gjknc\" (UID: \"7aba1f79-1ca8-483e-921c-fd1ab682d9e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" Sep 09 02:33:08.289531 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.289343 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlgcr\" (UniqueName: \"kubernetes.io/projected/7aba1f79-1ca8-483e-921c-fd1ab682d9e5-kube-api-access-jlgcr\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-gjknc\" (UID: \"7aba1f79-1ca8-483e-921c-fd1ab682d9e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" Sep 09 02:33:08.289531 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.289365 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0cbe25b-3f1b-4c95-85ab-a05da38d1490-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b9skq\" (UID: \"a0cbe25b-3f1b-4c95-85ab-a05da38d1490\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" Sep 09 02:33:08.289531 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.289385 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv9tb\" (UniqueName: \"kubernetes.io/projected/7afcb052-d8ca-4a7e-9d85-0fcce7270162-kube-api-access-fv9tb\") pod \"limitador-operator-controller-manager-74495c69f8-r69dh\" (UID: \"7afcb052-d8ca-4a7e-9d85-0fcce7270162\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh" Sep 09 02:33:08.289865 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.289846 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aba1f79-1ca8-483e-921c-fd1ab682d9e5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-gjknc\" (UID: \"7aba1f79-1ca8-483e-921c-fd1ab682d9e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" Sep 09 02:33:08.289949 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.289851 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0cbe25b-3f1b-4c95-85ab-a05da38d1490-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b9skq\" (UID: \"a0cbe25b-3f1b-4c95-85ab-a05da38d1490\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" Sep 09 02:33:08.298597 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.298454 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv9tb\" (UniqueName: \"kubernetes.io/projected/7afcb052-d8ca-4a7e-9d85-0fcce7270162-kube-api-access-fv9tb\") pod \"limitador-operator-controller-manager-74495c69f8-r69dh\" (UID: \"7afcb052-d8ca-4a7e-9d85-0fcce7270162\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh" Sep 09 02:33:08.299028 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.299005 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlgcr\" (UniqueName: \"kubernetes.io/projected/7aba1f79-1ca8-483e-921c-fd1ab682d9e5-kube-api-access-jlgcr\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-gjknc\" (UID: \"7aba1f79-1ca8-483e-921c-fd1ab682d9e5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" Sep 09 02:33:08.299138 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.299008 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh9bx\" (UniqueName: \"kubernetes.io/projected/a0cbe25b-3f1b-4c95-85ab-a05da38d1490-kube-api-access-nh9bx\") pod \"kuadrant-operator-controller-manager-76f8f686bf-b9skq\" (UID: \"a0cbe25b-3f1b-4c95-85ab-a05da38d1490\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" Sep 09 02:33:08.339400 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.339377 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:08.341443 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.341420 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:33:08.341770 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.341751 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:08.343902 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.343879 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7nf75\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:33:08.345768 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.345747 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:08.347760 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.347740 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7nf75\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:33:08.389910 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.389877 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f4abbc8-223d-4381-b5d8-cdcf22ca890a-extensions-socket-volume\") pod \"4f4abbc8-223d-4381-b5d8-cdcf22ca890a\" (UID: \"4f4abbc8-223d-4381-b5d8-cdcf22ca890a\") " Sep 09 02:33:08.389910 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.389914 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsmnn\" (UniqueName: \"kubernetes.io/projected/b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd-kube-api-access-lsmnn\") pod \"b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd\" (UID: \"b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd\") " Sep 09 02:33:08.390162 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.389959 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qzxc\" (UniqueName: \"kubernetes.io/projected/4f4abbc8-223d-4381-b5d8-cdcf22ca890a-kube-api-access-5qzxc\") pod \"4f4abbc8-223d-4381-b5d8-cdcf22ca890a\" (UID: \"4f4abbc8-223d-4381-b5d8-cdcf22ca890a\") " Sep 09 02:33:08.390370 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.390340 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f4abbc8-223d-4381-b5d8-cdcf22ca890a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4f4abbc8-223d-4381-b5d8-cdcf22ca890a" (UID: "4f4abbc8-223d-4381-b5d8-cdcf22ca890a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 02:33:08.392146 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.392120 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd-kube-api-access-lsmnn" (OuterVolumeSpecName: "kube-api-access-lsmnn") pod "b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" (UID: "b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd"). InnerVolumeSpecName "kube-api-access-lsmnn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:33:08.392146 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.392135 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f4abbc8-223d-4381-b5d8-cdcf22ca890a-kube-api-access-5qzxc" (OuterVolumeSpecName: "kube-api-access-5qzxc") pod "4f4abbc8-223d-4381-b5d8-cdcf22ca890a" (UID: "4f4abbc8-223d-4381-b5d8-cdcf22ca890a"). InnerVolumeSpecName "kube-api-access-5qzxc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:33:08.427544 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.427515 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:08.491285 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.491248 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f4abbc8-223d-4381-b5d8-cdcf22ca890a-extensions-socket-volume\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:33:08.491285 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.491276 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsmnn\" (UniqueName: \"kubernetes.io/projected/b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd-kube-api-access-lsmnn\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:33:08.491285 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.491293 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qzxc\" (UniqueName: \"kubernetes.io/projected/4f4abbc8-223d-4381-b5d8-cdcf22ca890a-kube-api-access-5qzxc\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:33:08.513736 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.513688 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" Sep 09 02:33:08.521751 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.521725 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" Sep 09 02:33:08.523395 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.523369 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:08.528451 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.528426 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh" Sep 09 02:33:08.694556 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.694520 2577 generic.go:356] "Generic (PLEG): container finished" podID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" containerID="65e27342454515f67ac88e9653391323d8a7390337d8cc7fbe8c2589c570e598" exitCode=0 Sep 09 02:33:08.694756 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.694567 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:33:08.694756 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.694619 2577 scope.go:117] "RemoveContainer" containerID="65e27342454515f67ac88e9653391323d8a7390337d8cc7fbe8c2589c570e598" Sep 09 02:33:08.696261 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.696238 2577 generic.go:356] "Generic (PLEG): container finished" podID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" containerID="98cdf30c23ea6c418f5bf9e727f2537b93f0036b799ad18c809fc7e8d3a2404e" exitCode=0 Sep 09 02:33:08.696381 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.696277 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:08.696600 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.696579 2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" container="manager" Sep 09 02:33:08.697450 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.697420 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:08.699470 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.699442 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7nf75\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:33:08.701511 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.701481 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:08.703463 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.703441 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7nf75\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:33:08.704811 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.704795 2577 scope.go:117] "RemoveContainer" containerID="65e27342454515f67ac88e9653391323d8a7390337d8cc7fbe8c2589c570e598" Sep 09 02:33:08.705176 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.705150 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65e27342454515f67ac88e9653391323d8a7390337d8cc7fbe8c2589c570e598\": container with ID starting with 65e27342454515f67ac88e9653391323d8a7390337d8cc7fbe8c2589c570e598 not found: ID does not exist" containerID="65e27342454515f67ac88e9653391323d8a7390337d8cc7fbe8c2589c570e598" Sep 09 02:33:08.705252 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.705190 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65e27342454515f67ac88e9653391323d8a7390337d8cc7fbe8c2589c570e598"} err="failed to get container status \"65e27342454515f67ac88e9653391323d8a7390337d8cc7fbe8c2589c570e598\": rpc error: code = NotFound desc = could not find container \"65e27342454515f67ac88e9653391323d8a7390337d8cc7fbe8c2589c570e598\": container with ID starting with 65e27342454515f67ac88e9653391323d8a7390337d8cc7fbe8c2589c570e598 not found: ID does not exist" Sep 09 02:33:08.705252 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.705213 2577 scope.go:117] "RemoveContainer" containerID="98cdf30c23ea6c418f5bf9e727f2537b93f0036b799ad18c809fc7e8d3a2404e" Sep 09 02:33:08.705369 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.705288 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-7cx57\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="da40869d-0781-4a15-b26b-a7163daa8eb1" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:08.711274 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.711249 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:08.713098 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.713072 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7nf75\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:33:08.713844 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.713827 2577 scope.go:117] "RemoveContainer" containerID="98cdf30c23ea6c418f5bf9e727f2537b93f0036b799ad18c809fc7e8d3a2404e" Sep 09 02:33:08.714114 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.714094 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98cdf30c23ea6c418f5bf9e727f2537b93f0036b799ad18c809fc7e8d3a2404e\": container with ID starting with 98cdf30c23ea6c418f5bf9e727f2537b93f0036b799ad18c809fc7e8d3a2404e not found: ID does not exist" containerID="98cdf30c23ea6c418f5bf9e727f2537b93f0036b799ad18c809fc7e8d3a2404e" Sep 09 02:33:08.714159 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.714121 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98cdf30c23ea6c418f5bf9e727f2537b93f0036b799ad18c809fc7e8d3a2404e"} err="failed to get container status \"98cdf30c23ea6c418f5bf9e727f2537b93f0036b799ad18c809fc7e8d3a2404e\": rpc error: code = NotFound desc = could not find container \"98cdf30c23ea6c418f5bf9e727f2537b93f0036b799ad18c809fc7e8d3a2404e\": container with ID starting with 98cdf30c23ea6c418f5bf9e727f2537b93f0036b799ad18c809fc7e8d3a2404e not found: ID does not exist" Sep 09 02:33:08.715030 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.715009 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-7cx57\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="da40869d-0781-4a15-b26b-a7163daa8eb1" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:08.716867 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.716846 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-drn6v\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-drn6v" Sep 09 02:33:08.718775 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.718750 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-7nf75\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-7nf75" Sep 09 02:33:08.720664 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:08.720621 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-7cx57\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="da40869d-0781-4a15-b26b-a7163daa8eb1" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:08.805834 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.805805 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc"] Sep 09 02:33:08.808540 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:33:08.808510 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aba1f79_1ca8_483e_921c_fd1ab682d9e5.slice/crio-1db1372864c3a7353beeec4fc5eeeec0242a6514039ae5abf1ebe3474fd4feb6 WatchSource:0}: Error finding container 1db1372864c3a7353beeec4fc5eeeec0242a6514039ae5abf1ebe3474fd4feb6: Status 404 returned error can't find the container with id 1db1372864c3a7353beeec4fc5eeeec0242a6514039ae5abf1ebe3474fd4feb6 Sep 09 02:33:08.817652 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.817597 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh"] Sep 09 02:33:08.821558 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:33:08.821535 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7afcb052_d8ca_4a7e_9d85_0fcce7270162.slice/crio-3057d81e4c5efba636725df00fc74016bb28fa43d1f063a81bdec747d172d470 WatchSource:0}: Error finding container 3057d81e4c5efba636725df00fc74016bb28fa43d1f063a81bdec747d172d470: Status 404 returned error can't find the container with id 3057d81e4c5efba636725df00fc74016bb28fa43d1f063a81bdec747d172d470 Sep 09 02:33:08.821959 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:08.821931 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq"] Sep 09 02:33:08.823985 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:33:08.823959 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0cbe25b_3f1b_4c95_85ab_a05da38d1490.slice/crio-7486dfa8ca9cf90b8e0683245f540cafb1ec1496886d3da5df8825ea11c49041 WatchSource:0}: Error finding container 7486dfa8ca9cf90b8e0683245f540cafb1ec1496886d3da5df8825ea11c49041: Status 404 returned error can't find the container with id 7486dfa8ca9cf90b8e0683245f540cafb1ec1496886d3da5df8825ea11c49041 Sep 09 02:33:09.587467 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.587435 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f4abbc8-223d-4381-b5d8-cdcf22ca890a" path="/var/lib/kubelet/pods/4f4abbc8-223d-4381-b5d8-cdcf22ca890a/volumes" Sep 09 02:33:09.587868 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.587797 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd" path="/var/lib/kubelet/pods/b7135472-7ebf-4cc8-b91f-eb4fc9ddc0bd/volumes" Sep 09 02:33:09.704980 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.704941 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" podUID="da40869d-0781-4a15-b26b-a7163daa8eb1" containerName="manager" containerID="cri-o://b68226cd8169b7306755e54fb24d1eb5cbf25bd19799154ce267b937ffc2c474" gracePeriod=2 Sep 09 02:33:09.706673 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.706640 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" event={"ID":"7aba1f79-1ca8-483e-921c-fd1ab682d9e5","Type":"ContainerStarted","Data":"cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa"} Sep 09 02:33:09.706809 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.706696 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" event={"ID":"7aba1f79-1ca8-483e-921c-fd1ab682d9e5","Type":"ContainerStarted","Data":"1db1372864c3a7353beeec4fc5eeeec0242a6514039ae5abf1ebe3474fd4feb6"} Sep 09 02:33:09.706809 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.706747 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" Sep 09 02:33:09.709585 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.709549 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh" event={"ID":"7afcb052-d8ca-4a7e-9d85-0fcce7270162","Type":"ContainerStarted","Data":"a1af8fbf8bac923065e8049bb54e733aa0452ce055f94af0b48dfc77e4aaf742"} Sep 09 02:33:09.709734 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.709623 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh" Sep 09 02:33:09.709734 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.709674 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh" event={"ID":"7afcb052-d8ca-4a7e-9d85-0fcce7270162","Type":"ContainerStarted","Data":"3057d81e4c5efba636725df00fc74016bb28fa43d1f063a81bdec747d172d470"} Sep 09 02:33:09.711584 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.711492 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" event={"ID":"a0cbe25b-3f1b-4c95-85ab-a05da38d1490","Type":"ContainerStarted","Data":"2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993"} Sep 09 02:33:09.711584 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.711539 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" Sep 09 02:33:09.711584 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.711557 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" event={"ID":"a0cbe25b-3f1b-4c95-85ab-a05da38d1490","Type":"ContainerStarted","Data":"7486dfa8ca9cf90b8e0683245f540cafb1ec1496886d3da5df8825ea11c49041"} Sep 09 02:33:09.727992 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.727947 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" podStartSLOduration=1.727932359 podStartE2EDuration="1.727932359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:33:09.726002856 +0000 UTC m=+458.668571771" watchObservedRunningTime="2026-09-09 02:33:09.727932359 +0000 UTC m=+458.670501269" Sep 09 02:33:09.746166 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.746120 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh" podStartSLOduration=1.7461053199999998 podStartE2EDuration="1.74610532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:33:09.744881331 +0000 UTC m=+458.687450241" watchObservedRunningTime="2026-09-09 02:33:09.74610532 +0000 UTC m=+458.688674230" Sep 09 02:33:09.763424 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.763376 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" podStartSLOduration=1.763361258 podStartE2EDuration="1.763361258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:33:09.762769678 +0000 UTC m=+458.705338618" watchObservedRunningTime="2026-09-09 02:33:09.763361258 +0000 UTC m=+458.705930167" Sep 09 02:33:09.962657 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:09.962617 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:10.005885 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:10.005851 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rw7vm\" (UniqueName: \"kubernetes.io/projected/da40869d-0781-4a15-b26b-a7163daa8eb1-kube-api-access-rw7vm\") pod \"da40869d-0781-4a15-b26b-a7163daa8eb1\" (UID: \"da40869d-0781-4a15-b26b-a7163daa8eb1\") " Sep 09 02:33:10.006026 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:10.005922 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da40869d-0781-4a15-b26b-a7163daa8eb1-extensions-socket-volume\") pod \"da40869d-0781-4a15-b26b-a7163daa8eb1\" (UID: \"da40869d-0781-4a15-b26b-a7163daa8eb1\") " Sep 09 02:33:10.006357 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:10.006334 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da40869d-0781-4a15-b26b-a7163daa8eb1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "da40869d-0781-4a15-b26b-a7163daa8eb1" (UID: "da40869d-0781-4a15-b26b-a7163daa8eb1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 02:33:10.008105 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:10.008079 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da40869d-0781-4a15-b26b-a7163daa8eb1-kube-api-access-rw7vm" (OuterVolumeSpecName: "kube-api-access-rw7vm") pod "da40869d-0781-4a15-b26b-a7163daa8eb1" (UID: "da40869d-0781-4a15-b26b-a7163daa8eb1"). InnerVolumeSpecName "kube-api-access-rw7vm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:33:10.107271 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:10.107188 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/da40869d-0781-4a15-b26b-a7163daa8eb1-extensions-socket-volume\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:33:10.107271 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:10.107217 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw7vm\" (UniqueName: \"kubernetes.io/projected/da40869d-0781-4a15-b26b-a7163daa8eb1-kube-api-access-rw7vm\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:33:10.719159 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:10.719118 2577 generic.go:356] "Generic (PLEG): container finished" podID="da40869d-0781-4a15-b26b-a7163daa8eb1" containerID="b68226cd8169b7306755e54fb24d1eb5cbf25bd19799154ce267b937ffc2c474" exitCode=0 Sep 09 02:33:10.719763 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:10.719165 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:10.719763 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:10.719212 2577 scope.go:117] "RemoveContainer" containerID="b68226cd8169b7306755e54fb24d1eb5cbf25bd19799154ce267b937ffc2c474" Sep 09 02:33:10.721703 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:10.721679 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-7cx57\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="da40869d-0781-4a15-b26b-a7163daa8eb1" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:10.727828 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:10.727792 2577 scope.go:117] "RemoveContainer" containerID="b68226cd8169b7306755e54fb24d1eb5cbf25bd19799154ce267b937ffc2c474" Sep 09 02:33:10.728070 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:10.728049 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b68226cd8169b7306755e54fb24d1eb5cbf25bd19799154ce267b937ffc2c474\": container with ID starting with b68226cd8169b7306755e54fb24d1eb5cbf25bd19799154ce267b937ffc2c474 not found: ID does not exist" containerID="b68226cd8169b7306755e54fb24d1eb5cbf25bd19799154ce267b937ffc2c474" Sep 09 02:33:10.728117 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:10.728076 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b68226cd8169b7306755e54fb24d1eb5cbf25bd19799154ce267b937ffc2c474"} err="failed to get container status \"b68226cd8169b7306755e54fb24d1eb5cbf25bd19799154ce267b937ffc2c474\": rpc error: code = NotFound desc = could not find container \"b68226cd8169b7306755e54fb24d1eb5cbf25bd19799154ce267b937ffc2c474\": container with ID starting with b68226cd8169b7306755e54fb24d1eb5cbf25bd19799154ce267b937ffc2c474 not found: ID does not exist" Sep 09 02:33:10.729523 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:10.729497 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-7cx57\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="da40869d-0781-4a15-b26b-a7163daa8eb1" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:10.999298 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:10.999258 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:11.111304 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:11.111261 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:11.580515 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:11.580478 2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-7cx57\" is forbidden: User \"system:node:ip-10-0-128-153.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-153.ec2.internal' and this object" podUID="da40869d-0781-4a15-b26b-a7163daa8eb1" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-7cx57" Sep 09 02:33:11.582559 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:11.582535 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da40869d-0781-4a15-b26b-a7163daa8eb1" path="/var/lib/kubelet/pods/da40869d-0781-4a15-b26b-a7163daa8eb1/volumes" Sep 09 02:33:13.460915 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:13.460872 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:13.521041 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:13.521007 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:15.412903 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:15.412864 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:15.478045 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:15.478012 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:16.395081 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:16.395047 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:16.446435 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:16.446398 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:16.789224 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:16.789186 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:16.889851 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:16.889818 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:17.191292 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:17.191203 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:17.292601 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:17.292568 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:20.721356 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:20.721319 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-r69dh" Sep 09 02:33:20.721776 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:20.721407 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" Sep 09 02:33:20.721776 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:20.721545 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" Sep 09 02:33:20.778101 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:20.778068 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq"] Sep 09 02:33:20.778327 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:20.778302 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" podUID="a0cbe25b-3f1b-4c95-85ab-a05da38d1490" containerName="manager" containerID="cri-o://2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993" gracePeriod=10 Sep 09 02:33:21.154342 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.154308 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r"] Sep 09 02:33:21.165166 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.165138 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" Sep 09 02:33:21.174347 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.174318 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r"] Sep 09 02:33:21.303463 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.303372 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e4c07e1-0dd9-48df-96ea-6533468d146a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-sbn4r\" (UID: \"4e4c07e1-0dd9-48df-96ea-6533468d146a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" Sep 09 02:33:21.303463 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.303426 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5xlh\" (UniqueName: \"kubernetes.io/projected/4e4c07e1-0dd9-48df-96ea-6533468d146a-kube-api-access-z5xlh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-sbn4r\" (UID: \"4e4c07e1-0dd9-48df-96ea-6533468d146a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" Sep 09 02:33:21.404821 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.404779 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e4c07e1-0dd9-48df-96ea-6533468d146a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-sbn4r\" (UID: \"4e4c07e1-0dd9-48df-96ea-6533468d146a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" Sep 09 02:33:21.404976 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.404836 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5xlh\" (UniqueName: \"kubernetes.io/projected/4e4c07e1-0dd9-48df-96ea-6533468d146a-kube-api-access-z5xlh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-sbn4r\" (UID: \"4e4c07e1-0dd9-48df-96ea-6533468d146a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" Sep 09 02:33:21.405154 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.405127 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e4c07e1-0dd9-48df-96ea-6533468d146a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-sbn4r\" (UID: \"4e4c07e1-0dd9-48df-96ea-6533468d146a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" Sep 09 02:33:21.414193 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.414161 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5xlh\" (UniqueName: \"kubernetes.io/projected/4e4c07e1-0dd9-48df-96ea-6533468d146a-kube-api-access-z5xlh\") pod \"kuadrant-operator-controller-manager-5769f99d9f-sbn4r\" (UID: \"4e4c07e1-0dd9-48df-96ea-6533468d146a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" Sep 09 02:33:21.476939 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.476907 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" Sep 09 02:33:21.565302 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.565235 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" Sep 09 02:33:21.707447 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.707410 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0cbe25b-3f1b-4c95-85ab-a05da38d1490-extensions-socket-volume\") pod \"a0cbe25b-3f1b-4c95-85ab-a05da38d1490\" (UID: \"a0cbe25b-3f1b-4c95-85ab-a05da38d1490\") " Sep 09 02:33:21.707447 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.707451 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh9bx\" (UniqueName: \"kubernetes.io/projected/a0cbe25b-3f1b-4c95-85ab-a05da38d1490-kube-api-access-nh9bx\") pod \"a0cbe25b-3f1b-4c95-85ab-a05da38d1490\" (UID: \"a0cbe25b-3f1b-4c95-85ab-a05da38d1490\") " Sep 09 02:33:21.707872 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.707852 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0cbe25b-3f1b-4c95-85ab-a05da38d1490-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a0cbe25b-3f1b-4c95-85ab-a05da38d1490" (UID: "a0cbe25b-3f1b-4c95-85ab-a05da38d1490"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 02:33:21.709561 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.709540 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0cbe25b-3f1b-4c95-85ab-a05da38d1490-kube-api-access-nh9bx" (OuterVolumeSpecName: "kube-api-access-nh9bx") pod "a0cbe25b-3f1b-4c95-85ab-a05da38d1490" (UID: "a0cbe25b-3f1b-4c95-85ab-a05da38d1490"). InnerVolumeSpecName "kube-api-access-nh9bx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:33:21.770619 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.770579 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r"] Sep 09 02:33:21.773541 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:33:21.773515 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4c07e1_0dd9_48df_96ea_6533468d146a.slice/crio-e3ca4e3314741e1888bd6a0828ecc55a9aa351dc1ce96c6fd651576c9e728480 WatchSource:0}: Error finding container e3ca4e3314741e1888bd6a0828ecc55a9aa351dc1ce96c6fd651576c9e728480: Status 404 returned error can't find the container with id e3ca4e3314741e1888bd6a0828ecc55a9aa351dc1ce96c6fd651576c9e728480 Sep 09 02:33:21.776457 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.776431 2577 generic.go:356] "Generic (PLEG): container finished" podID="a0cbe25b-3f1b-4c95-85ab-a05da38d1490" containerID="2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993" exitCode=0 Sep 09 02:33:21.776538 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.776473 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" Sep 09 02:33:21.776584 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.776474 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" event={"ID":"a0cbe25b-3f1b-4c95-85ab-a05da38d1490","Type":"ContainerDied","Data":"2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993"} Sep 09 02:33:21.776640 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.776612 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq" event={"ID":"a0cbe25b-3f1b-4c95-85ab-a05da38d1490","Type":"ContainerDied","Data":"7486dfa8ca9cf90b8e0683245f540cafb1ec1496886d3da5df8825ea11c49041"} Sep 09 02:33:21.776734 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.776668 2577 scope.go:117] "RemoveContainer" containerID="2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993" Sep 09 02:33:21.786210 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.786066 2577 scope.go:117] "RemoveContainer" containerID="2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993" Sep 09 02:33:21.786353 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:21.786335 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993\": container with ID starting with 2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993 not found: ID does not exist" containerID="2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993" Sep 09 02:33:21.786396 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.786361 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993"} err="failed to get container status \"2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993\": rpc error: code = NotFound desc = could not find container \"2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993\": container with ID starting with 2b13b2ae5546bcef09e64cde80911216d5c355178cccaf9524c167af3c7c9993 not found: ID does not exist" Sep 09 02:33:21.810061 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.809024 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq"] Sep 09 02:33:21.810061 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.809844 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0cbe25b-3f1b-4c95-85ab-a05da38d1490-extensions-socket-volume\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:33:21.810061 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.809875 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh9bx\" (UniqueName: \"kubernetes.io/projected/a0cbe25b-3f1b-4c95-85ab-a05da38d1490-kube-api-access-nh9bx\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:33:21.817050 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:21.817027 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-b9skq"] Sep 09 02:33:22.786709 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:22.786669 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" event={"ID":"4e4c07e1-0dd9-48df-96ea-6533468d146a","Type":"ContainerStarted","Data":"4024800b0af6f175497066c73a301751e6ed27f6a9111dc50d538009aaf150e6"} Sep 09 02:33:22.787089 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:22.786867 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" Sep 09 02:33:22.787089 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:22.786901 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" event={"ID":"4e4c07e1-0dd9-48df-96ea-6533468d146a","Type":"ContainerStarted","Data":"e3ca4e3314741e1888bd6a0828ecc55a9aa351dc1ce96c6fd651576c9e728480"} Sep 09 02:33:22.809105 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:22.809044 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" podStartSLOduration=1.809022298 podStartE2EDuration="1.809022298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:33:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:33:22.808239818 +0000 UTC m=+471.750808728" watchObservedRunningTime="2026-09-09 02:33:22.809022298 +0000 UTC m=+471.751591209" Sep 09 02:33:23.581952 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:23.581913 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0cbe25b-3f1b-4c95-85ab-a05da38d1490" path="/var/lib/kubelet/pods/a0cbe25b-3f1b-4c95-85ab-a05da38d1490/volumes" Sep 09 02:33:25.205614 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:25.205580 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:25.345143 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:25.345074 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:28.201268 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:28.201235 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:28.302708 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:28.302675 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:33.793310 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:33.793277 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-sbn4r" Sep 09 02:33:33.840418 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:33.840381 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc"] Sep 09 02:33:33.840669 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:33.840638 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" podUID="7aba1f79-1ca8-483e-921c-fd1ab682d9e5" containerName="manager" containerID="cri-o://cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa" gracePeriod=10 Sep 09 02:33:34.088190 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.088159 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" Sep 09 02:33:34.212311 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.212271 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlgcr\" (UniqueName: \"kubernetes.io/projected/7aba1f79-1ca8-483e-921c-fd1ab682d9e5-kube-api-access-jlgcr\") pod \"7aba1f79-1ca8-483e-921c-fd1ab682d9e5\" (UID: \"7aba1f79-1ca8-483e-921c-fd1ab682d9e5\") " Sep 09 02:33:34.212464 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.212373 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aba1f79-1ca8-483e-921c-fd1ab682d9e5-extensions-socket-volume\") pod \"7aba1f79-1ca8-483e-921c-fd1ab682d9e5\" (UID: \"7aba1f79-1ca8-483e-921c-fd1ab682d9e5\") " Sep 09 02:33:34.212822 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.212793 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aba1f79-1ca8-483e-921c-fd1ab682d9e5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7aba1f79-1ca8-483e-921c-fd1ab682d9e5" (UID: "7aba1f79-1ca8-483e-921c-fd1ab682d9e5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 02:33:34.214473 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.214454 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aba1f79-1ca8-483e-921c-fd1ab682d9e5-kube-api-access-jlgcr" (OuterVolumeSpecName: "kube-api-access-jlgcr") pod "7aba1f79-1ca8-483e-921c-fd1ab682d9e5" (UID: "7aba1f79-1ca8-483e-921c-fd1ab682d9e5"). InnerVolumeSpecName "kube-api-access-jlgcr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:33:34.313032 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.312987 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlgcr\" (UniqueName: \"kubernetes.io/projected/7aba1f79-1ca8-483e-921c-fd1ab682d9e5-kube-api-access-jlgcr\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:33:34.313032 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.313029 2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aba1f79-1ca8-483e-921c-fd1ab682d9e5-extensions-socket-volume\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:33:34.856737 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.856698 2577 generic.go:356] "Generic (PLEG): container finished" podID="7aba1f79-1ca8-483e-921c-fd1ab682d9e5" containerID="cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa" exitCode=0 Sep 09 02:33:34.857207 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.856750 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" Sep 09 02:33:34.857207 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.856770 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" event={"ID":"7aba1f79-1ca8-483e-921c-fd1ab682d9e5","Type":"ContainerDied","Data":"cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa"} Sep 09 02:33:34.857207 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.856812 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc" event={"ID":"7aba1f79-1ca8-483e-921c-fd1ab682d9e5","Type":"ContainerDied","Data":"1db1372864c3a7353beeec4fc5eeeec0242a6514039ae5abf1ebe3474fd4feb6"} Sep 09 02:33:34.857207 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.856834 2577 scope.go:117] "RemoveContainer" containerID="cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa" Sep 09 02:33:34.866250 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.866232 2577 scope.go:117] "RemoveContainer" containerID="cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa" Sep 09 02:33:34.866550 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:33:34.866528 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa\": container with ID starting with cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa not found: ID does not exist" containerID="cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa" Sep 09 02:33:34.866598 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.866560 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa"} err="failed to get container status \"cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa\": rpc error: code = NotFound desc = could not find container \"cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa\": container with ID starting with cfb7e380c64897ed383312d482fe124b7173fde54c9d506b645a217e7f071dfa not found: ID does not exist" Sep 09 02:33:34.879232 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.879205 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc"] Sep 09 02:33:34.883155 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:34.883131 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-gjknc"] Sep 09 02:33:35.582337 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:35.582301 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aba1f79-1ca8-483e-921c-fd1ab682d9e5" path="/var/lib/kubelet/pods/7aba1f79-1ca8-483e-921c-fd1ab682d9e5/volumes" Sep 09 02:33:37.137977 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.137938 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9"] Sep 09 02:33:37.138346 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.138284 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0cbe25b-3f1b-4c95-85ab-a05da38d1490" containerName="manager" Sep 09 02:33:37.138346 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.138295 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0cbe25b-3f1b-4c95-85ab-a05da38d1490" containerName="manager" Sep 09 02:33:37.138346 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.138310 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aba1f79-1ca8-483e-921c-fd1ab682d9e5" containerName="manager" Sep 09 02:33:37.138346 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.138316 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aba1f79-1ca8-483e-921c-fd1ab682d9e5" containerName="manager" Sep 09 02:33:37.138489 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.138380 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0cbe25b-3f1b-4c95-85ab-a05da38d1490" containerName="manager" Sep 09 02:33:37.138489 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.138388 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aba1f79-1ca8-483e-921c-fd1ab682d9e5" containerName="manager" Sep 09 02:33:37.142663 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.142640 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.145161 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.145116 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 09 02:33:37.145309 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.145232 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-747m2\"" Sep 09 02:33:37.155773 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.155750 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9"] Sep 09 02:33:37.158945 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.158917 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:37.240055 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.240020 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.240055 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.240069 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7198324c-df6f-42f5-addd-827e1021a416-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.240272 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.240138 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.240272 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.240172 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7198324c-df6f-42f5-addd-827e1021a416-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.240272 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.240201 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcl62\" (UniqueName: \"kubernetes.io/projected/7198324c-df6f-42f5-addd-827e1021a416-kube-api-access-bcl62\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.240272 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.240235 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.240272 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.240270 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.240431 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.240301 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7198324c-df6f-42f5-addd-827e1021a416-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.240431 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.240321 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.341387 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.341349 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7198324c-df6f-42f5-addd-827e1021a416-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.341387 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.341391 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.341659 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.341431 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.341659 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.341458 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7198324c-df6f-42f5-addd-827e1021a416-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.341659 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.341485 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.341659 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.341508 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7198324c-df6f-42f5-addd-827e1021a416-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.341659 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.341618 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcl62\" (UniqueName: \"kubernetes.io/projected/7198324c-df6f-42f5-addd-827e1021a416-kube-api-access-bcl62\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.341921 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.341701 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.341921 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.341736 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.342040 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.342021 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.342094 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.342078 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.342139 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.342117 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.342206 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.342186 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.342337 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.342313 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7198324c-df6f-42f5-addd-827e1021a416-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.343913 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.343889 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7198324c-df6f-42f5-addd-827e1021a416-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.344296 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.344276 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7198324c-df6f-42f5-addd-827e1021a416-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.349209 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.349187 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7198324c-df6f-42f5-addd-827e1021a416-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.349518 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.349496 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcl62\" (UniqueName: \"kubernetes.io/projected/7198324c-df6f-42f5-addd-827e1021a416-kube-api-access-bcl62\") pod \"maas-default-gateway-openshift-default-5ccb968c66-x8mr9\" (UID: \"7198324c-df6f-42f5-addd-827e1021a416\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.453925 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.453838 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:37.457741 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.457704 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:37.728255 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.728226 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9"] Sep 09 02:33:37.732229 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:33:37.732196 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7198324c_df6f_42f5_addd_827e1021a416.slice/crio-3ef5e3e843448f27bfc9e346cbc78b017b5c041b32caa57592ad4f1ea52ea4b0 WatchSource:0}: Error finding container 3ef5e3e843448f27bfc9e346cbc78b017b5c041b32caa57592ad4f1ea52ea4b0: Status 404 returned error can't find the container with id 3ef5e3e843448f27bfc9e346cbc78b017b5c041b32caa57592ad4f1ea52ea4b0 Sep 09 02:33:37.874026 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:37.873987 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" event={"ID":"7198324c-df6f-42f5-addd-827e1021a416","Type":"ContainerStarted","Data":"3ef5e3e843448f27bfc9e346cbc78b017b5c041b32caa57592ad4f1ea52ea4b0"} Sep 09 02:33:41.701753 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:41.701709 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 02:33:41.702094 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:41.701791 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 02:33:41.702094 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:41.701821 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 02:33:41.895856 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:41.895811 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" event={"ID":"7198324c-df6f-42f5-addd-827e1021a416","Type":"ContainerStarted","Data":"c5423d5e560ada9dcb79394e9313b414a8925e3c69f8d0360e6a2689deccf499"} Sep 09 02:33:41.914604 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:41.914552 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" podStartSLOduration=0.947584656 podStartE2EDuration="4.914536212s" totalImagesPullingTime="3.966951556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:33:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:33:41.912712938 +0000 UTC m=+490.855281861" watchObservedRunningTime="2026-09-09 02:33:41.914536212 +0000 UTC m=+490.857105122" Sep 09 02:33:42.454553 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:42.454521 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:42.458747 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:42.458722 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:42.901011 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:42.900977 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:42.902072 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:42.902053 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:33:50.349160 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:50.349119 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:50.449867 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:50.449834 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:51.626352 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.626312 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:51.632105 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.632070 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4b8vq"] Sep 09 02:33:51.635835 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.635813 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" Sep 09 02:33:51.638332 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.638309 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 09 02:33:51.642743 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.642718 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4b8vq"] Sep 09 02:33:51.661977 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.661944 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx8zc\" (UniqueName: \"kubernetes.io/projected/a63f26a4-d879-43e5-8720-d15a69bd2292-kube-api-access-bx8zc\") pod \"limitador-limitador-8588749977-4b8vq\" (UID: \"a63f26a4-d879-43e5-8720-d15a69bd2292\") " pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" Sep 09 02:33:51.662136 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.662041 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a63f26a4-d879-43e5-8720-d15a69bd2292-config-file\") pod \"limitador-limitador-8588749977-4b8vq\" (UID: \"a63f26a4-d879-43e5-8720-d15a69bd2292\") " pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" Sep 09 02:33:51.727223 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.727187 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4b8vq"] Sep 09 02:33:51.737580 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.737547 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:51.762826 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.762797 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx8zc\" (UniqueName: \"kubernetes.io/projected/a63f26a4-d879-43e5-8720-d15a69bd2292-kube-api-access-bx8zc\") pod \"limitador-limitador-8588749977-4b8vq\" (UID: \"a63f26a4-d879-43e5-8720-d15a69bd2292\") " pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" Sep 09 02:33:51.762974 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.762885 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a63f26a4-d879-43e5-8720-d15a69bd2292-config-file\") pod \"limitador-limitador-8588749977-4b8vq\" (UID: \"a63f26a4-d879-43e5-8720-d15a69bd2292\") " pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" Sep 09 02:33:51.763532 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.763513 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a63f26a4-d879-43e5-8720-d15a69bd2292-config-file\") pod \"limitador-limitador-8588749977-4b8vq\" (UID: \"a63f26a4-d879-43e5-8720-d15a69bd2292\") " pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" Sep 09 02:33:51.770314 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.770292 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx8zc\" (UniqueName: \"kubernetes.io/projected/a63f26a4-d879-43e5-8720-d15a69bd2292-kube-api-access-bx8zc\") pod \"limitador-limitador-8588749977-4b8vq\" (UID: \"a63f26a4-d879-43e5-8720-d15a69bd2292\") " pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" Sep 09 02:33:51.796494 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.796457 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:51.947278 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:51.947193 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" Sep 09 02:33:52.215691 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:52.215663 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4b8vq"] Sep 09 02:33:52.217994 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:33:52.217964 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda63f26a4_d879_43e5_8720_d15a69bd2292.slice/crio-efff7e7c928d7131144b6ba155acb9e12498ced94b28cee9b2cdd4ec0dfd1b7f WatchSource:0}: Error finding container efff7e7c928d7131144b6ba155acb9e12498ced94b28cee9b2cdd4ec0dfd1b7f: Status 404 returned error can't find the container with id efff7e7c928d7131144b6ba155acb9e12498ced94b28cee9b2cdd4ec0dfd1b7f Sep 09 02:33:52.436852 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:52.436810 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:52.515334 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:52.515302 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:33:52.519076 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:52.519050 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-hjqnf"] Sep 09 02:33:52.523580 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:52.523558 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-hjqnf" Sep 09 02:33:52.526078 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:52.526058 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l9pnp\"" Sep 09 02:33:52.526993 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:52.526973 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-hjqnf"] Sep 09 02:33:52.569551 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:52.569514 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l75l6\" (UniqueName: \"kubernetes.io/projected/85efb283-1967-435f-8a92-d19f8ed38de3-kube-api-access-l75l6\") pod \"authorino-59c974bd48-hjqnf\" (UID: \"85efb283-1967-435f-8a92-d19f8ed38de3\") " pod="kuadrant-system/authorino-59c974bd48-hjqnf" Sep 09 02:33:52.671069 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:52.671031 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l75l6\" (UniqueName: \"kubernetes.io/projected/85efb283-1967-435f-8a92-d19f8ed38de3-kube-api-access-l75l6\") pod \"authorino-59c974bd48-hjqnf\" (UID: \"85efb283-1967-435f-8a92-d19f8ed38de3\") " pod="kuadrant-system/authorino-59c974bd48-hjqnf" Sep 09 02:33:52.678890 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:52.678857 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l75l6\" (UniqueName: \"kubernetes.io/projected/85efb283-1967-435f-8a92-d19f8ed38de3-kube-api-access-l75l6\") pod \"authorino-59c974bd48-hjqnf\" (UID: \"85efb283-1967-435f-8a92-d19f8ed38de3\") " pod="kuadrant-system/authorino-59c974bd48-hjqnf" Sep 09 02:33:52.834786 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:52.834684 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-hjqnf" Sep 09 02:33:52.948791 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:52.948760 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" event={"ID":"a63f26a4-d879-43e5-8720-d15a69bd2292","Type":"ContainerStarted","Data":"efff7e7c928d7131144b6ba155acb9e12498ced94b28cee9b2cdd4ec0dfd1b7f"} Sep 09 02:33:53.103156 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:53.103124 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-hjqnf"] Sep 09 02:33:53.105971 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:33:53.105937 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85efb283_1967_435f_8a92_d19f8ed38de3.slice/crio-c3f0f9185c41b3b4b2baabe32d45d1776975f3a7fb8e4f344f31aa75e31552b5 WatchSource:0}: Error finding container c3f0f9185c41b3b4b2baabe32d45d1776975f3a7fb8e4f344f31aa75e31552b5: Status 404 returned error can't find the container with id c3f0f9185c41b3b4b2baabe32d45d1776975f3a7fb8e4f344f31aa75e31552b5 Sep 09 02:33:53.955083 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:53.955043 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" event={"ID":"a63f26a4-d879-43e5-8720-d15a69bd2292","Type":"ContainerStarted","Data":"c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1"} Sep 09 02:33:53.955522 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:53.955186 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" Sep 09 02:33:53.956310 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:53.956285 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-hjqnf" event={"ID":"85efb283-1967-435f-8a92-d19f8ed38de3","Type":"ContainerStarted","Data":"c3f0f9185c41b3b4b2baabe32d45d1776975f3a7fb8e4f344f31aa75e31552b5"} Sep 09 02:33:53.972042 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:53.971966 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" podStartSLOduration=1.815121693 podStartE2EDuration="2.971949792s" totalImagesPullingTime="1.156828099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:33:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:33:53.969531472 +0000 UTC m=+502.912100396" watchObservedRunningTime="2026-09-09 02:33:53.971949792 +0000 UTC m=+502.914518702" Sep 09 02:33:55.967497 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:55.967400 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-hjqnf" event={"ID":"85efb283-1967-435f-8a92-d19f8ed38de3","Type":"ContainerStarted","Data":"58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db"} Sep 09 02:33:55.980859 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:33:55.980807 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-hjqnf" podStartSLOduration=1.463417029 podStartE2EDuration="3.980789858s" totalImagesPullingTime="2.517372829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:33:55.980103427 +0000 UTC m=+504.922672337" watchObservedRunningTime="2026-09-09 02:33:55.980789858 +0000 UTC m=+504.923358767" Sep 09 02:34:04.962215 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:04.962180 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" Sep 09 02:34:06.249126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:06.249090 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4b8vq"] Sep 09 02:34:06.249542 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:06.249283 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" podUID="a63f26a4-d879-43e5-8720-d15a69bd2292" containerName="limitador" containerID="cri-o://c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1" gracePeriod=30 Sep 09 02:34:06.802217 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:06.802191 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" Sep 09 02:34:06.897497 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:06.897398 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a63f26a4-d879-43e5-8720-d15a69bd2292-config-file\") pod \"a63f26a4-d879-43e5-8720-d15a69bd2292\" (UID: \"a63f26a4-d879-43e5-8720-d15a69bd2292\") " Sep 09 02:34:06.897497 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:06.897476 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx8zc\" (UniqueName: \"kubernetes.io/projected/a63f26a4-d879-43e5-8720-d15a69bd2292-kube-api-access-bx8zc\") pod \"a63f26a4-d879-43e5-8720-d15a69bd2292\" (UID: \"a63f26a4-d879-43e5-8720-d15a69bd2292\") " Sep 09 02:34:06.897820 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:06.897793 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a63f26a4-d879-43e5-8720-d15a69bd2292-config-file" (OuterVolumeSpecName: "config-file") pod "a63f26a4-d879-43e5-8720-d15a69bd2292" (UID: "a63f26a4-d879-43e5-8720-d15a69bd2292"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 02:34:06.899565 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:06.899536 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a63f26a4-d879-43e5-8720-d15a69bd2292-kube-api-access-bx8zc" (OuterVolumeSpecName: "kube-api-access-bx8zc") pod "a63f26a4-d879-43e5-8720-d15a69bd2292" (UID: "a63f26a4-d879-43e5-8720-d15a69bd2292"). InnerVolumeSpecName "kube-api-access-bx8zc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:34:06.998514 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:06.998482 2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a63f26a4-d879-43e5-8720-d15a69bd2292-config-file\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:34:06.998514 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:06.998511 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx8zc\" (UniqueName: \"kubernetes.io/projected/a63f26a4-d879-43e5-8720-d15a69bd2292-kube-api-access-bx8zc\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:34:07.023193 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:07.023154 2577 generic.go:356] "Generic (PLEG): container finished" podID="a63f26a4-d879-43e5-8720-d15a69bd2292" containerID="c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1" exitCode=0 Sep 09 02:34:07.023193 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:07.023196 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" Sep 09 02:34:07.023422 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:07.023233 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" event={"ID":"a63f26a4-d879-43e5-8720-d15a69bd2292","Type":"ContainerDied","Data":"c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1"} Sep 09 02:34:07.023422 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:07.023282 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-4b8vq" event={"ID":"a63f26a4-d879-43e5-8720-d15a69bd2292","Type":"ContainerDied","Data":"efff7e7c928d7131144b6ba155acb9e12498ced94b28cee9b2cdd4ec0dfd1b7f"} Sep 09 02:34:07.023422 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:07.023313 2577 scope.go:117] "RemoveContainer" containerID="c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1" Sep 09 02:34:07.033083 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:07.033064 2577 scope.go:117] "RemoveContainer" containerID="c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1" Sep 09 02:34:07.033364 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:34:07.033346 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1\": container with ID starting with c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1 not found: ID does not exist" containerID="c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1" Sep 09 02:34:07.033410 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:07.033373 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1"} err="failed to get container status \"c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1\": rpc error: code = NotFound desc = could not find container \"c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1\": container with ID starting with c5c1042858705608fac7520213f8f18d310193e587af8b8d878040652414c2d1 not found: ID does not exist" Sep 09 02:34:07.044005 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:07.043981 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4b8vq"] Sep 09 02:34:07.049980 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:07.049956 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-4b8vq"] Sep 09 02:34:07.581593 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:07.581556 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a63f26a4-d879-43e5-8720-d15a69bd2292" path="/var/lib/kubelet/pods/a63f26a4-d879-43e5-8720-d15a69bd2292/volumes" Sep 09 02:34:09.363201 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.363139 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:09.461765 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.461729 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:09.813207 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.813131 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4862n"] Sep 09 02:34:09.814554 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.814524 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a63f26a4-d879-43e5-8720-d15a69bd2292" containerName="limitador" Sep 09 02:34:09.814554 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.814547 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a63f26a4-d879-43e5-8720-d15a69bd2292" containerName="limitador" Sep 09 02:34:09.814759 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.814672 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a63f26a4-d879-43e5-8720-d15a69bd2292" containerName="limitador" Sep 09 02:34:09.819166 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.819144 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4862n"] Sep 09 02:34:09.819264 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.819225 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" Sep 09 02:34:09.822672 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.822605 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 09 02:34:09.822780 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.822701 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 09 02:34:09.822780 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.822756 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5q5d8\"" Sep 09 02:34:09.823457 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.823441 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 09 02:34:09.837831 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.837805 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:09.923625 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.923595 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/14768d61-1723-4768-b9cb-2801d3966763-data\") pod \"postgres-69fb4594cf-4862n\" (UID: \"14768d61-1723-4768-b9cb-2801d3966763\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" Sep 09 02:34:09.923804 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:09.923656 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsjrw\" (UniqueName: \"kubernetes.io/projected/14768d61-1723-4768-b9cb-2801d3966763-kube-api-access-jsjrw\") pod \"postgres-69fb4594cf-4862n\" (UID: \"14768d61-1723-4768-b9cb-2801d3966763\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" Sep 09 02:34:10.024167 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:10.024128 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/14768d61-1723-4768-b9cb-2801d3966763-data\") pod \"postgres-69fb4594cf-4862n\" (UID: \"14768d61-1723-4768-b9cb-2801d3966763\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" Sep 09 02:34:10.024167 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:10.024168 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsjrw\" (UniqueName: \"kubernetes.io/projected/14768d61-1723-4768-b9cb-2801d3966763-kube-api-access-jsjrw\") pod \"postgres-69fb4594cf-4862n\" (UID: \"14768d61-1723-4768-b9cb-2801d3966763\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" Sep 09 02:34:10.024585 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:10.024559 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/14768d61-1723-4768-b9cb-2801d3966763-data\") pod \"postgres-69fb4594cf-4862n\" (UID: \"14768d61-1723-4768-b9cb-2801d3966763\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" Sep 09 02:34:10.032233 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:10.032207 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsjrw\" (UniqueName: \"kubernetes.io/projected/14768d61-1723-4768-b9cb-2801d3966763-kube-api-access-jsjrw\") pod \"postgres-69fb4594cf-4862n\" (UID: \"14768d61-1723-4768-b9cb-2801d3966763\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" Sep 09 02:34:10.134366 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:10.134277 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" Sep 09 02:34:10.254204 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:10.254164 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:10.307133 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:10.307070 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:10.407032 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:10.407006 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-4862n"] Sep 09 02:34:10.408987 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:34:10.408958 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14768d61_1723_4768_b9cb_2801d3966763.slice/crio-ade51be2ba5154f6bfc105112b601dccba2cc5895a47547478549365caa03101 WatchSource:0}: Error finding container ade51be2ba5154f6bfc105112b601dccba2cc5895a47547478549365caa03101: Status 404 returned error can't find the container with id ade51be2ba5154f6bfc105112b601dccba2cc5895a47547478549365caa03101 Sep 09 02:34:11.043743 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:11.043702 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" event={"ID":"14768d61-1723-4768-b9cb-2801d3966763","Type":"ContainerStarted","Data":"ade51be2ba5154f6bfc105112b601dccba2cc5895a47547478549365caa03101"} Sep 09 02:34:14.060948 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:14.060913 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" event={"ID":"14768d61-1723-4768-b9cb-2801d3966763","Type":"ContainerStarted","Data":"972893d78eca6ea5b9c47d9badef4ae6de87e60f6caed370d4aa1984a89f1b2c"} Sep 09 02:34:14.061386 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:14.061041 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" Sep 09 02:34:14.077223 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:14.077175 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" podStartSLOduration=1.763320828 podStartE2EDuration="5.077159165s" totalImagesPullingTime="3.313838337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:34:14.075708343 +0000 UTC m=+523.018277253" watchObservedRunningTime="2026-09-09 02:34:14.077159165 +0000 UTC m=+523.019728074" Sep 09 02:34:20.096017 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.095987 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-4862n" Sep 09 02:34:20.404273 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.404191 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:20.520515 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.520480 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:20.680615 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.680538 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-966bcf9b8-jr6cm"] Sep 09 02:34:20.688919 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.688888 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-966bcf9b8-jr6cm"] Sep 09 02:34:20.689067 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.688993 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-966bcf9b8-jr6cm" Sep 09 02:34:20.691349 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.691325 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Sep 09 02:34:20.710053 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.710027 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-966bcf9b8-jr6cm"] Sep 09 02:34:20.710292 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:34:20.710274 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-sgnx9 tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-sgnx9 tls-cert]: context canceled" pod="kuadrant-system/authorino-966bcf9b8-jr6cm" podUID="0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0" Sep 09 02:34:20.732382 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.732357 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-4c69w"] Sep 09 02:34:20.736139 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.736120 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-4c69w" Sep 09 02:34:20.742765 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.742732 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-4c69w"] Sep 09 02:34:20.820458 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.820421 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0-tls-cert\") pod \"authorino-966bcf9b8-jr6cm\" (UID: \"0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0\") " pod="kuadrant-system/authorino-966bcf9b8-jr6cm" Sep 09 02:34:20.820458 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.820463 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2165fca7-03b2-4555-88ca-e2a7cc2e3e70-tls-cert\") pod \"authorino-6d4dfc649f-4c69w\" (UID: \"2165fca7-03b2-4555-88ca-e2a7cc2e3e70\") " pod="kuadrant-system/authorino-6d4dfc649f-4c69w" Sep 09 02:34:20.820712 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.820485 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2bxz\" (UniqueName: \"kubernetes.io/projected/2165fca7-03b2-4555-88ca-e2a7cc2e3e70-kube-api-access-n2bxz\") pod \"authorino-6d4dfc649f-4c69w\" (UID: \"2165fca7-03b2-4555-88ca-e2a7cc2e3e70\") " pod="kuadrant-system/authorino-6d4dfc649f-4c69w" Sep 09 02:34:20.820712 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.820553 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgnx9\" (UniqueName: \"kubernetes.io/projected/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0-kube-api-access-sgnx9\") pod \"authorino-966bcf9b8-jr6cm\" (UID: \"0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0\") " pod="kuadrant-system/authorino-966bcf9b8-jr6cm" Sep 09 02:34:20.913576 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.913539 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-4c69w"] Sep 09 02:34:20.913849 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:34:20.913828 2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-n2bxz tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-4c69w" podUID="2165fca7-03b2-4555-88ca-e2a7cc2e3e70" Sep 09 02:34:20.920926 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.920892 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgnx9\" (UniqueName: \"kubernetes.io/projected/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0-kube-api-access-sgnx9\") pod \"authorino-966bcf9b8-jr6cm\" (UID: \"0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0\") " pod="kuadrant-system/authorino-966bcf9b8-jr6cm" Sep 09 02:34:20.921059 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.920977 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0-tls-cert\") pod \"authorino-966bcf9b8-jr6cm\" (UID: \"0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0\") " pod="kuadrant-system/authorino-966bcf9b8-jr6cm" Sep 09 02:34:20.921059 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.921002 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2165fca7-03b2-4555-88ca-e2a7cc2e3e70-tls-cert\") pod \"authorino-6d4dfc649f-4c69w\" (UID: \"2165fca7-03b2-4555-88ca-e2a7cc2e3e70\") " pod="kuadrant-system/authorino-6d4dfc649f-4c69w" Sep 09 02:34:20.921059 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.921026 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2bxz\" (UniqueName: \"kubernetes.io/projected/2165fca7-03b2-4555-88ca-e2a7cc2e3e70-kube-api-access-n2bxz\") pod \"authorino-6d4dfc649f-4c69w\" (UID: \"2165fca7-03b2-4555-88ca-e2a7cc2e3e70\") " pod="kuadrant-system/authorino-6d4dfc649f-4c69w" Sep 09 02:34:20.923509 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.923478 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0-tls-cert\") pod \"authorino-966bcf9b8-jr6cm\" (UID: \"0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0\") " pod="kuadrant-system/authorino-966bcf9b8-jr6cm" Sep 09 02:34:20.923726 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.923705 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2165fca7-03b2-4555-88ca-e2a7cc2e3e70-tls-cert\") pod \"authorino-6d4dfc649f-4c69w\" (UID: \"2165fca7-03b2-4555-88ca-e2a7cc2e3e70\") " pod="kuadrant-system/authorino-6d4dfc649f-4c69w" Sep 09 02:34:20.933058 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.932997 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgnx9\" (UniqueName: \"kubernetes.io/projected/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0-kube-api-access-sgnx9\") pod \"authorino-966bcf9b8-jr6cm\" (UID: \"0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0\") " pod="kuadrant-system/authorino-966bcf9b8-jr6cm" Sep 09 02:34:20.934959 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.934935 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2bxz\" (UniqueName: \"kubernetes.io/projected/2165fca7-03b2-4555-88ca-e2a7cc2e3e70-kube-api-access-n2bxz\") pod \"authorino-6d4dfc649f-4c69w\" (UID: \"2165fca7-03b2-4555-88ca-e2a7cc2e3e70\") " pod="kuadrant-system/authorino-6d4dfc649f-4c69w" Sep 09 02:34:20.946718 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.946692 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-67db6955c5-26qzq"] Sep 09 02:34:20.950742 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.950723 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67db6955c5-26qzq" Sep 09 02:34:20.956839 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:20.956817 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67db6955c5-26qzq"] Sep 09 02:34:21.096191 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.096160 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-4c69w" Sep 09 02:34:21.096623 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.096160 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-966bcf9b8-jr6cm" Sep 09 02:34:21.103897 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.103876 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-4c69w" Sep 09 02:34:21.106903 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.106885 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-966bcf9b8-jr6cm" Sep 09 02:34:21.122415 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.122364 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/66831307-d7a7-4cba-9d79-74dac3615bf8-tls-cert\") pod \"authorino-67db6955c5-26qzq\" (UID: \"66831307-d7a7-4cba-9d79-74dac3615bf8\") " pod="kuadrant-system/authorino-67db6955c5-26qzq" Sep 09 02:34:21.122639 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.122537 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnsh2\" (UniqueName: \"kubernetes.io/projected/66831307-d7a7-4cba-9d79-74dac3615bf8-kube-api-access-gnsh2\") pod \"authorino-67db6955c5-26qzq\" (UID: \"66831307-d7a7-4cba-9d79-74dac3615bf8\") " pod="kuadrant-system/authorino-67db6955c5-26qzq" Sep 09 02:34:21.223899 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.223795 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2165fca7-03b2-4555-88ca-e2a7cc2e3e70-tls-cert\") pod \"2165fca7-03b2-4555-88ca-e2a7cc2e3e70\" (UID: \"2165fca7-03b2-4555-88ca-e2a7cc2e3e70\") " Sep 09 02:34:21.223899 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.223871 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sgnx9\" (UniqueName: \"kubernetes.io/projected/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0-kube-api-access-sgnx9\") pod \"0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0\" (UID: \"0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0\") " Sep 09 02:34:21.224166 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.223974 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0-tls-cert\") pod \"0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0\" (UID: \"0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0\") " Sep 09 02:34:21.224166 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.224097 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2bxz\" (UniqueName: \"kubernetes.io/projected/2165fca7-03b2-4555-88ca-e2a7cc2e3e70-kube-api-access-n2bxz\") pod \"2165fca7-03b2-4555-88ca-e2a7cc2e3e70\" (UID: \"2165fca7-03b2-4555-88ca-e2a7cc2e3e70\") " Sep 09 02:34:21.224304 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.224282 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnsh2\" (UniqueName: \"kubernetes.io/projected/66831307-d7a7-4cba-9d79-74dac3615bf8-kube-api-access-gnsh2\") pod \"authorino-67db6955c5-26qzq\" (UID: \"66831307-d7a7-4cba-9d79-74dac3615bf8\") " pod="kuadrant-system/authorino-67db6955c5-26qzq" Sep 09 02:34:21.224395 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.224314 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/66831307-d7a7-4cba-9d79-74dac3615bf8-tls-cert\") pod \"authorino-67db6955c5-26qzq\" (UID: \"66831307-d7a7-4cba-9d79-74dac3615bf8\") " pod="kuadrant-system/authorino-67db6955c5-26qzq" Sep 09 02:34:21.226201 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.226169 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0" (UID: "0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:34:21.226201 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.226186 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0-kube-api-access-sgnx9" (OuterVolumeSpecName: "kube-api-access-sgnx9") pod "0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0" (UID: "0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0"). InnerVolumeSpecName "kube-api-access-sgnx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:34:21.226392 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.226291 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2165fca7-03b2-4555-88ca-e2a7cc2e3e70-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2165fca7-03b2-4555-88ca-e2a7cc2e3e70" (UID: "2165fca7-03b2-4555-88ca-e2a7cc2e3e70"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:34:21.226693 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.226671 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2165fca7-03b2-4555-88ca-e2a7cc2e3e70-kube-api-access-n2bxz" (OuterVolumeSpecName: "kube-api-access-n2bxz") pod "2165fca7-03b2-4555-88ca-e2a7cc2e3e70" (UID: "2165fca7-03b2-4555-88ca-e2a7cc2e3e70"). InnerVolumeSpecName "kube-api-access-n2bxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:34:21.226856 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.226838 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/66831307-d7a7-4cba-9d79-74dac3615bf8-tls-cert\") pod \"authorino-67db6955c5-26qzq\" (UID: \"66831307-d7a7-4cba-9d79-74dac3615bf8\") " pod="kuadrant-system/authorino-67db6955c5-26qzq" Sep 09 02:34:21.232266 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.232243 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnsh2\" (UniqueName: \"kubernetes.io/projected/66831307-d7a7-4cba-9d79-74dac3615bf8-kube-api-access-gnsh2\") pod \"authorino-67db6955c5-26qzq\" (UID: \"66831307-d7a7-4cba-9d79-74dac3615bf8\") " pod="kuadrant-system/authorino-67db6955c5-26qzq" Sep 09 02:34:21.261885 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.261851 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67db6955c5-26qzq" Sep 09 02:34:21.325651 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.325357 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2bxz\" (UniqueName: \"kubernetes.io/projected/2165fca7-03b2-4555-88ca-e2a7cc2e3e70-kube-api-access-n2bxz\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:34:21.325651 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.325392 2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2165fca7-03b2-4555-88ca-e2a7cc2e3e70-tls-cert\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:34:21.325651 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.325420 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sgnx9\" (UniqueName: \"kubernetes.io/projected/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0-kube-api-access-sgnx9\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:34:21.325651 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.325433 2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0-tls-cert\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:34:21.529275 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:21.529239 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-67db6955c5-26qzq"] Sep 09 02:34:21.532774 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:34:21.532741 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod66831307_d7a7_4cba_9d79_74dac3615bf8.slice/crio-ef39569136a3ba974ef52ea135ca8282d98c315b8ad513e6ff295abf9a284c3e WatchSource:0}: Error finding container ef39569136a3ba974ef52ea135ca8282d98c315b8ad513e6ff295abf9a284c3e: Status 404 returned error can't find the container with id ef39569136a3ba974ef52ea135ca8282d98c315b8ad513e6ff295abf9a284c3e Sep 09 02:34:22.104545 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:22.104524 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-966bcf9b8-jr6cm" Sep 09 02:34:22.104545 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:22.104530 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67db6955c5-26qzq" event={"ID":"66831307-d7a7-4cba-9d79-74dac3615bf8","Type":"ContainerStarted","Data":"ef39569136a3ba974ef52ea135ca8282d98c315b8ad513e6ff295abf9a284c3e"} Sep 09 02:34:22.104917 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:22.104659 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-4c69w" Sep 09 02:34:22.131430 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:22.131396 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-966bcf9b8-jr6cm"] Sep 09 02:34:22.137167 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:22.137137 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-966bcf9b8-jr6cm"] Sep 09 02:34:22.156776 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:22.156744 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-4c69w"] Sep 09 02:34:22.163183 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:22.163153 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-4c69w"] Sep 09 02:34:22.862625 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:22.862591 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:23.111251 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:23.111206 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67db6955c5-26qzq" event={"ID":"66831307-d7a7-4cba-9d79-74dac3615bf8","Type":"ContainerStarted","Data":"51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4"} Sep 09 02:34:23.126905 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:23.126811 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-67db6955c5-26qzq" podStartSLOduration=2.621610632 podStartE2EDuration="3.126794714s" totalImagesPullingTime="505.184082ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:34:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:34:23.125962694 +0000 UTC m=+532.068531604" watchObservedRunningTime="2026-09-09 02:34:23.126794714 +0000 UTC m=+532.069363625" Sep 09 02:34:23.149754 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:23.149723 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-hjqnf"] Sep 09 02:34:23.149962 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:23.149937 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-hjqnf" podUID="85efb283-1967-435f-8a92-d19f8ed38de3" containerName="authorino" containerID="cri-o://58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db" gracePeriod=30 Sep 09 02:34:23.410453 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:23.410428 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-hjqnf" Sep 09 02:34:23.545691 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:23.545646 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l75l6\" (UniqueName: \"kubernetes.io/projected/85efb283-1967-435f-8a92-d19f8ed38de3-kube-api-access-l75l6\") pod \"85efb283-1967-435f-8a92-d19f8ed38de3\" (UID: \"85efb283-1967-435f-8a92-d19f8ed38de3\") " Sep 09 02:34:23.547727 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:23.547706 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85efb283-1967-435f-8a92-d19f8ed38de3-kube-api-access-l75l6" (OuterVolumeSpecName: "kube-api-access-l75l6") pod "85efb283-1967-435f-8a92-d19f8ed38de3" (UID: "85efb283-1967-435f-8a92-d19f8ed38de3"). InnerVolumeSpecName "kube-api-access-l75l6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:34:23.581530 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:23.581494 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0" path="/var/lib/kubelet/pods/0ee3e08b-e459-44f9-a7f1-d33c2ff1bcd0/volumes" Sep 09 02:34:23.581866 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:23.581848 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2165fca7-03b2-4555-88ca-e2a7cc2e3e70" path="/var/lib/kubelet/pods/2165fca7-03b2-4555-88ca-e2a7cc2e3e70/volumes" Sep 09 02:34:23.646816 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:23.646711 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l75l6\" (UniqueName: \"kubernetes.io/projected/85efb283-1967-435f-8a92-d19f8ed38de3-kube-api-access-l75l6\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:34:24.116907 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:24.116874 2577 generic.go:356] "Generic (PLEG): container finished" podID="85efb283-1967-435f-8a92-d19f8ed38de3" containerID="58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db" exitCode=0 Sep 09 02:34:24.117293 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:24.116913 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-hjqnf" Sep 09 02:34:24.117293 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:24.116955 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-hjqnf" event={"ID":"85efb283-1967-435f-8a92-d19f8ed38de3","Type":"ContainerDied","Data":"58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db"} Sep 09 02:34:24.117293 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:24.116997 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-hjqnf" event={"ID":"85efb283-1967-435f-8a92-d19f8ed38de3","Type":"ContainerDied","Data":"c3f0f9185c41b3b4b2baabe32d45d1776975f3a7fb8e4f344f31aa75e31552b5"} Sep 09 02:34:24.117293 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:24.117022 2577 scope.go:117] "RemoveContainer" containerID="58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db" Sep 09 02:34:24.125659 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:24.125639 2577 scope.go:117] "RemoveContainer" containerID="58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db" Sep 09 02:34:24.125949 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:34:24.125930 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db\": container with ID starting with 58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db not found: ID does not exist" containerID="58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db" Sep 09 02:34:24.126027 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:24.125961 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db"} err="failed to get container status \"58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db\": rpc error: code = NotFound desc = could not find container \"58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db\": container with ID starting with 58b256bea96c2c2dd6040a5ec677e2287f923f81c7fbabe014e52f1756e9c2db not found: ID does not exist" Sep 09 02:34:24.133837 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:24.133815 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-hjqnf"] Sep 09 02:34:24.137057 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:24.137036 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-hjqnf"] Sep 09 02:34:25.585929 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:25.585889 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85efb283-1967-435f-8a92-d19f8ed38de3" path="/var/lib/kubelet/pods/85efb283-1967-435f-8a92-d19f8ed38de3/volumes" Sep 09 02:34:27.249853 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:27.249820 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:27.305942 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:27.305906 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:28.167340 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.167310 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:28.191961 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.191929 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-794b78d468-c2h29"] Sep 09 02:34:28.192300 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.192288 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85efb283-1967-435f-8a92-d19f8ed38de3" containerName="authorino" Sep 09 02:34:28.192353 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.192302 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="85efb283-1967-435f-8a92-d19f8ed38de3" containerName="authorino" Sep 09 02:34:28.192392 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.192369 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="85efb283-1967-435f-8a92-d19f8ed38de3" containerName="authorino" Sep 09 02:34:28.198090 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.198058 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:28.200732 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.200700 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 09 02:34:28.200914 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.200895 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-v4mkl\"" Sep 09 02:34:28.200988 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.200921 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 09 02:34:28.201746 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.201727 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 09 02:34:28.207873 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.207846 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-794b78d468-c2h29"] Sep 09 02:34:28.266965 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.266933 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:28.367243 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.367207 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:28.371374 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.371337 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-95bf97f48-7hcbc"] Sep 09 02:34:28.375780 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.375763 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:28.384075 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.384047 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-95bf97f48-7hcbc"] Sep 09 02:34:28.389309 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.389285 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-webhook-cert\") pod \"maas-controller-794b78d468-c2h29\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:28.389429 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.389314 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rf58\" (UniqueName: \"kubernetes.io/projected/1a15016a-1255-4fa4-bf43-34d897cb9488-kube-api-access-5rf58\") pod \"maas-controller-794b78d468-c2h29\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:28.389429 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.389343 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-metrics-certs\") pod \"maas-controller-794b78d468-c2h29\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:28.490284 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.490250 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d67mz\" (UniqueName: \"kubernetes.io/projected/c2ad3753-1412-4cc5-b4b4-a4e7402c2217-kube-api-access-d67mz\") pod \"maas-controller-95bf97f48-7hcbc\" (UID: \"c2ad3753-1412-4cc5-b4b4-a4e7402c2217\") " pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:28.490448 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.490305 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-webhook-cert\") pod \"maas-controller-794b78d468-c2h29\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:28.490448 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.490353 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rf58\" (UniqueName: \"kubernetes.io/projected/1a15016a-1255-4fa4-bf43-34d897cb9488-kube-api-access-5rf58\") pod \"maas-controller-794b78d468-c2h29\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:28.490448 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.490433 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2ad3753-1412-4cc5-b4b4-a4e7402c2217-metrics-certs\") pod \"maas-controller-95bf97f48-7hcbc\" (UID: \"c2ad3753-1412-4cc5-b4b4-a4e7402c2217\") " pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:28.490626 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.490463 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2ad3753-1412-4cc5-b4b4-a4e7402c2217-webhook-cert\") pod \"maas-controller-95bf97f48-7hcbc\" (UID: \"c2ad3753-1412-4cc5-b4b4-a4e7402c2217\") " pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:28.490626 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.490501 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-metrics-certs\") pod \"maas-controller-794b78d468-c2h29\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:28.490740 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:34:28.490689 2577 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 09 02:34:28.490778 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:34:28.490764 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-metrics-certs podName:1a15016a-1255-4fa4-bf43-34d897cb9488 nodeName:}" failed. No retries permitted until 2026-09-09 02:34:28.990744627 +0000 UTC m=+537.933313530 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-metrics-certs") pod "maas-controller-794b78d468-c2h29" (UID: "1a15016a-1255-4fa4-bf43-34d897cb9488") : secret "maas-controller-metrics-tls" not found Sep 09 02:34:28.492814 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.492794 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-webhook-cert\") pod \"maas-controller-794b78d468-c2h29\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:28.498543 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.498520 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rf58\" (UniqueName: \"kubernetes.io/projected/1a15016a-1255-4fa4-bf43-34d897cb9488-kube-api-access-5rf58\") pod \"maas-controller-794b78d468-c2h29\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:28.591597 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.591543 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d67mz\" (UniqueName: \"kubernetes.io/projected/c2ad3753-1412-4cc5-b4b4-a4e7402c2217-kube-api-access-d67mz\") pod \"maas-controller-95bf97f48-7hcbc\" (UID: \"c2ad3753-1412-4cc5-b4b4-a4e7402c2217\") " pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:28.591832 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.591623 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2ad3753-1412-4cc5-b4b4-a4e7402c2217-metrics-certs\") pod \"maas-controller-95bf97f48-7hcbc\" (UID: \"c2ad3753-1412-4cc5-b4b4-a4e7402c2217\") " pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:28.591832 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.591681 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2ad3753-1412-4cc5-b4b4-a4e7402c2217-webhook-cert\") pod \"maas-controller-95bf97f48-7hcbc\" (UID: \"c2ad3753-1412-4cc5-b4b4-a4e7402c2217\") " pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:28.591832 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:34:28.591768 2577 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 09 02:34:28.592005 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:34:28.591837 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2ad3753-1412-4cc5-b4b4-a4e7402c2217-metrics-certs podName:c2ad3753-1412-4cc5-b4b4-a4e7402c2217 nodeName:}" failed. No retries permitted until 2026-09-09 02:34:29.091819751 +0000 UTC m=+538.034388644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c2ad3753-1412-4cc5-b4b4-a4e7402c2217-metrics-certs") pod "maas-controller-95bf97f48-7hcbc" (UID: "c2ad3753-1412-4cc5-b4b4-a4e7402c2217") : secret "maas-controller-metrics-tls" not found Sep 09 02:34:28.594363 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.594331 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2ad3753-1412-4cc5-b4b4-a4e7402c2217-webhook-cert\") pod \"maas-controller-95bf97f48-7hcbc\" (UID: \"c2ad3753-1412-4cc5-b4b4-a4e7402c2217\") " pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:28.599492 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.599469 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d67mz\" (UniqueName: \"kubernetes.io/projected/c2ad3753-1412-4cc5-b4b4-a4e7402c2217-kube-api-access-d67mz\") pod \"maas-controller-95bf97f48-7hcbc\" (UID: \"c2ad3753-1412-4cc5-b4b4-a4e7402c2217\") " pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:28.770776 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.770689 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:28.995244 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.995203 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-metrics-certs\") pod \"maas-controller-794b78d468-c2h29\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:28.997745 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:28.997719 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-metrics-certs\") pod \"maas-controller-794b78d468-c2h29\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:29.095997 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:29.095906 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2ad3753-1412-4cc5-b4b4-a4e7402c2217-metrics-certs\") pod \"maas-controller-95bf97f48-7hcbc\" (UID: \"c2ad3753-1412-4cc5-b4b4-a4e7402c2217\") " pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:29.098241 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:29.098220 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c2ad3753-1412-4cc5-b4b4-a4e7402c2217-metrics-certs\") pod \"maas-controller-95bf97f48-7hcbc\" (UID: \"c2ad3753-1412-4cc5-b4b4-a4e7402c2217\") " pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:29.110243 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:29.110216 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:29.286774 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:29.286741 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:29.382052 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:29.382025 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-794b78d468-c2h29"] Sep 09 02:34:29.382823 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:34:29.382792 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a15016a_1255_4fa4_bf43_34d897cb9488.slice/crio-8068efcd50c627c89b4f2b6a204b385903ed514bb59cddb9641a73bf7c493caf WatchSource:0}: Error finding container 8068efcd50c627c89b4f2b6a204b385903ed514bb59cddb9641a73bf7c493caf: Status 404 returned error can't find the container with id 8068efcd50c627c89b4f2b6a204b385903ed514bb59cddb9641a73bf7c493caf Sep 09 02:34:29.557416 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:29.557370 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-95bf97f48-7hcbc"] Sep 09 02:34:29.559614 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:34:29.559578 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2ad3753_1412_4cc5_b4b4_a4e7402c2217.slice/crio-a581e828533f9f33ce905fdaeeef1d34fe5926fd79a243290f5a4b573a37ce4b WatchSource:0}: Error finding container a581e828533f9f33ce905fdaeeef1d34fe5926fd79a243290f5a4b573a37ce4b: Status 404 returned error can't find the container with id a581e828533f9f33ce905fdaeeef1d34fe5926fd79a243290f5a4b573a37ce4b Sep 09 02:34:30.155013 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:30.154925 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794b78d468-c2h29" event={"ID":"1a15016a-1255-4fa4-bf43-34d897cb9488","Type":"ContainerStarted","Data":"8068efcd50c627c89b4f2b6a204b385903ed514bb59cddb9641a73bf7c493caf"} Sep 09 02:34:30.156784 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:30.156751 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95bf97f48-7hcbc" event={"ID":"c2ad3753-1412-4cc5-b4b4-a4e7402c2217","Type":"ContainerStarted","Data":"a581e828533f9f33ce905fdaeeef1d34fe5926fd79a243290f5a4b573a37ce4b"} Sep 09 02:34:31.360901 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:31.360865 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:31.460267 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:31.460169 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:32.167587 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:32.167549 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95bf97f48-7hcbc" event={"ID":"c2ad3753-1412-4cc5-b4b4-a4e7402c2217","Type":"ContainerStarted","Data":"5c110535db390195c17bd0dee98dd55a011b75fd2238f993a5819d3c26e04a41"} Sep 09 02:34:32.167791 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:32.167691 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:32.169017 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:32.168994 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794b78d468-c2h29" event={"ID":"1a15016a-1255-4fa4-bf43-34d897cb9488","Type":"ContainerStarted","Data":"166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95"} Sep 09 02:34:32.169157 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:32.169142 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:32.186797 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:32.186745 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-95bf97f48-7hcbc" podStartSLOduration=2.578620311 podStartE2EDuration="4.186729885s" totalImagesPullingTime="1.608109574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:34:32.184824733 +0000 UTC m=+541.127393674" watchObservedRunningTime="2026-09-09 02:34:32.186729885 +0000 UTC m=+541.129298795" Sep 09 02:34:32.203994 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:32.203936 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-794b78d468-c2h29" podStartSLOduration=2.422926001 podStartE2EDuration="4.203918547s" totalImagesPullingTime="1.780992546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:34:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:34:32.202159494 +0000 UTC m=+541.144728413" watchObservedRunningTime="2026-09-09 02:34:32.203918547 +0000 UTC m=+541.146487457" Sep 09 02:34:43.177086 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.177057 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-95bf97f48-7hcbc" Sep 09 02:34:43.177457 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.177125 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:43.236865 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.236825 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-794b78d468-c2h29"] Sep 09 02:34:43.237127 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.237097 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-794b78d468-c2h29" podUID="1a15016a-1255-4fa4-bf43-34d897cb9488" containerName="manager" containerID="cri-o://166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95" gracePeriod=30 Sep 09 02:34:43.487904 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.487877 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:43.506617 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.506500 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rf58\" (UniqueName: \"kubernetes.io/projected/1a15016a-1255-4fa4-bf43-34d897cb9488-kube-api-access-5rf58\") pod \"1a15016a-1255-4fa4-bf43-34d897cb9488\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " Sep 09 02:34:43.506617 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.506550 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-metrics-certs\") pod \"1a15016a-1255-4fa4-bf43-34d897cb9488\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " Sep 09 02:34:43.507788 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.507066 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-webhook-cert\") pod \"1a15016a-1255-4fa4-bf43-34d897cb9488\" (UID: \"1a15016a-1255-4fa4-bf43-34d897cb9488\") " Sep 09 02:34:43.509797 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.509748 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a15016a-1255-4fa4-bf43-34d897cb9488-kube-api-access-5rf58" (OuterVolumeSpecName: "kube-api-access-5rf58") pod "1a15016a-1255-4fa4-bf43-34d897cb9488" (UID: "1a15016a-1255-4fa4-bf43-34d897cb9488"). InnerVolumeSpecName "kube-api-access-5rf58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:34:43.510995 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.510964 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1a15016a-1255-4fa4-bf43-34d897cb9488" (UID: "1a15016a-1255-4fa4-bf43-34d897cb9488"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:34:43.511205 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.511182 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1a15016a-1255-4fa4-bf43-34d897cb9488" (UID: "1a15016a-1255-4fa4-bf43-34d897cb9488"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:34:43.608415 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.608378 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rf58\" (UniqueName: \"kubernetes.io/projected/1a15016a-1255-4fa4-bf43-34d897cb9488-kube-api-access-5rf58\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:34:43.608415 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.608410 2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-metrics-certs\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:34:43.608674 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:43.608424 2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1a15016a-1255-4fa4-bf43-34d897cb9488-webhook-cert\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:34:44.232461 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:44.232423 2577 generic.go:356] "Generic (PLEG): container finished" podID="1a15016a-1255-4fa4-bf43-34d897cb9488" containerID="166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95" exitCode=0 Sep 09 02:34:44.232959 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:44.232472 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-794b78d468-c2h29" Sep 09 02:34:44.232959 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:44.232511 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794b78d468-c2h29" event={"ID":"1a15016a-1255-4fa4-bf43-34d897cb9488","Type":"ContainerDied","Data":"166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95"} Sep 09 02:34:44.232959 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:44.232570 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-794b78d468-c2h29" event={"ID":"1a15016a-1255-4fa4-bf43-34d897cb9488","Type":"ContainerDied","Data":"8068efcd50c627c89b4f2b6a204b385903ed514bb59cddb9641a73bf7c493caf"} Sep 09 02:34:44.232959 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:44.232593 2577 scope.go:117] "RemoveContainer" containerID="166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95" Sep 09 02:34:44.241358 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:44.241185 2577 scope.go:117] "RemoveContainer" containerID="166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95" Sep 09 02:34:44.241490 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:34:44.241473 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95\": container with ID starting with 166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95 not found: ID does not exist" containerID="166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95" Sep 09 02:34:44.241540 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:44.241496 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95"} err="failed to get container status \"166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95\": rpc error: code = NotFound desc = could not find container \"166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95\": container with ID starting with 166081f2fec571acd09d94cf5c5c429ecffcc9b6b9211622c4268b76f8fe7e95 not found: ID does not exist" Sep 09 02:34:44.252793 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:44.252766 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-794b78d468-c2h29"] Sep 09 02:34:44.256298 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:44.256274 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-794b78d468-c2h29"] Sep 09 02:34:45.581273 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:45.581230 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a15016a-1255-4fa4-bf43-34d897cb9488" path="/var/lib/kubelet/pods/1a15016a-1255-4fa4-bf43-34d897cb9488/volumes" Sep 09 02:34:58.824349 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:58.824312 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:34:58.879147 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:34:58.879114 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:35:00.571713 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:00.571678 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:35:00.665762 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:00.665728 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:35:01.983779 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:01.983734 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:35:02.084346 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.084308 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:35:02.124271 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.124239 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w"] Sep 09 02:35:02.124654 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.124640 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a15016a-1255-4fa4-bf43-34d897cb9488" containerName="manager" Sep 09 02:35:02.124708 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.124655 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a15016a-1255-4fa4-bf43-34d897cb9488" containerName="manager" Sep 09 02:35:02.124745 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.124727 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a15016a-1255-4fa4-bf43-34d897cb9488" containerName="manager" Sep 09 02:35:02.127868 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.127850 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.130285 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.130239 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 09 02:35:02.130430 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.130249 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-d2m9r\"" Sep 09 02:35:02.130430 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.130329 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 09 02:35:02.138675 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.138647 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w"] Sep 09 02:35:02.168376 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.168340 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pgjj\" (UniqueName: \"kubernetes.io/projected/79c43c00-6038-4adf-8b64-899ecd155a34-kube-api-access-7pgjj\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.168560 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.168455 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c43c00-6038-4adf-8b64-899ecd155a34-openshift-service-ca\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.168560 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.168484 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c43c00-6038-4adf-8b64-899ecd155a34-metrics-certs\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.168560 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.168518 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79c43c00-6038-4adf-8b64-899ecd155a34-maas-api-tls\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.269593 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.269552 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c43c00-6038-4adf-8b64-899ecd155a34-openshift-service-ca\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.269593 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.269592 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c43c00-6038-4adf-8b64-899ecd155a34-metrics-certs\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.269893 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.269616 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79c43c00-6038-4adf-8b64-899ecd155a34-maas-api-tls\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.269893 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.269707 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pgjj\" (UniqueName: \"kubernetes.io/projected/79c43c00-6038-4adf-8b64-899ecd155a34-kube-api-access-7pgjj\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.269893 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:35:02.269752 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found Sep 09 02:35:02.269893 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:35:02.269833 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c43c00-6038-4adf-8b64-899ecd155a34-metrics-certs podName:79c43c00-6038-4adf-8b64-899ecd155a34 nodeName:}" failed. No retries permitted until 2026-09-09 02:35:02.769815043 +0000 UTC m=+571.712383953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79c43c00-6038-4adf-8b64-899ecd155a34-metrics-certs") pod "maas-api-5dfc675bbf-fcp2w" (UID: "79c43c00-6038-4adf-8b64-899ecd155a34") : secret "maas-api-metrics-tls" not found Sep 09 02:35:02.270285 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.270251 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79c43c00-6038-4adf-8b64-899ecd155a34-openshift-service-ca\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.272203 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.272180 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79c43c00-6038-4adf-8b64-899ecd155a34-maas-api-tls\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.278285 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.278262 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pgjj\" (UniqueName: \"kubernetes.io/projected/79c43c00-6038-4adf-8b64-899ecd155a34-kube-api-access-7pgjj\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.292550 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.292523 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:35:02.474309 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.474268 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:35:02.574287 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.574189 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:35:02.627033 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.626997 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-47qh8"] Sep 09 02:35:02.631025 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.630992 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" Sep 09 02:35:02.634329 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.634291 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 09 02:35:02.634481 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.634403 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4chv6\"" Sep 09 02:35:02.665079 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.665048 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-47qh8"] Sep 09 02:35:02.673489 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.673449 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54e7834d-b125-48e4-88be-860e40a5bfdb-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-47qh8\" (UID: \"54e7834d-b125-48e4-88be-860e40a5bfdb\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" Sep 09 02:35:02.673691 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.673534 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-675km\" (UniqueName: \"kubernetes.io/projected/54e7834d-b125-48e4-88be-860e40a5bfdb-kube-api-access-675km\") pod \"payload-processing-5b54b7fbb7-47qh8\" (UID: \"54e7834d-b125-48e4-88be-860e40a5bfdb\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" Sep 09 02:35:02.775017 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.774986 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-675km\" (UniqueName: \"kubernetes.io/projected/54e7834d-b125-48e4-88be-860e40a5bfdb-kube-api-access-675km\") pod \"payload-processing-5b54b7fbb7-47qh8\" (UID: \"54e7834d-b125-48e4-88be-860e40a5bfdb\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" Sep 09 02:35:02.775193 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.775168 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c43c00-6038-4adf-8b64-899ecd155a34-metrics-certs\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:02.775290 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.775220 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54e7834d-b125-48e4-88be-860e40a5bfdb-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-47qh8\" (UID: \"54e7834d-b125-48e4-88be-860e40a5bfdb\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" Sep 09 02:35:02.775350 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:35:02.775326 2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found Sep 09 02:35:02.775423 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:35:02.775408 2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79c43c00-6038-4adf-8b64-899ecd155a34-metrics-certs podName:79c43c00-6038-4adf-8b64-899ecd155a34 nodeName:}" failed. No retries permitted until 2026-09-09 02:35:03.775385376 +0000 UTC m=+572.717954264 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/79c43c00-6038-4adf-8b64-899ecd155a34-metrics-certs") pod "maas-api-5dfc675bbf-fcp2w" (UID: "79c43c00-6038-4adf-8b64-899ecd155a34") : secret "maas-api-metrics-tls" not found Sep 09 02:35:02.776151 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.776126 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54e7834d-b125-48e4-88be-860e40a5bfdb-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-47qh8\" (UID: \"54e7834d-b125-48e4-88be-860e40a5bfdb\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" Sep 09 02:35:02.783968 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.783929 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-675km\" (UniqueName: \"kubernetes.io/projected/54e7834d-b125-48e4-88be-860e40a5bfdb-kube-api-access-675km\") pod \"payload-processing-5b54b7fbb7-47qh8\" (UID: \"54e7834d-b125-48e4-88be-860e40a5bfdb\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" Sep 09 02:35:02.848470 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.848353 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:35:02.945066 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:02.945026 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" Sep 09 02:35:03.225647 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:03.225604 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-47qh8"] Sep 09 02:35:03.227526 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:35:03.227502 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54e7834d_b125_48e4_88be_860e40a5bfdb.slice/crio-7387839d7b90304b322cc5490b39e89963b63b2d722afc981fe5d4a9562a132c WatchSource:0}: Error finding container 7387839d7b90304b322cc5490b39e89963b63b2d722afc981fe5d4a9562a132c: Status 404 returned error can't find the container with id 7387839d7b90304b322cc5490b39e89963b63b2d722afc981fe5d4a9562a132c Sep 09 02:35:03.331757 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:03.331722 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" event={"ID":"54e7834d-b125-48e4-88be-860e40a5bfdb","Type":"ContainerStarted","Data":"7387839d7b90304b322cc5490b39e89963b63b2d722afc981fe5d4a9562a132c"} Sep 09 02:35:03.793654 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:03.793161 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c43c00-6038-4adf-8b64-899ecd155a34-metrics-certs\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:03.797655 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:03.797274 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79c43c00-6038-4adf-8b64-899ecd155a34-metrics-certs\") pod \"maas-api-5dfc675bbf-fcp2w\" (UID: \"79c43c00-6038-4adf-8b64-899ecd155a34\") " pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:03.940334 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:03.940297 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:04.255525 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:04.255495 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w"] Sep 09 02:35:04.340448 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:04.340383 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" event={"ID":"79c43c00-6038-4adf-8b64-899ecd155a34","Type":"ContainerStarted","Data":"2d29d48acd58886457a7c3dbba4894acf8a4d574a6c1d39e4c58f5a659e63c7f"} Sep 09 02:35:06.355079 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:06.355044 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" event={"ID":"79c43c00-6038-4adf-8b64-899ecd155a34","Type":"ContainerStarted","Data":"2bccc7e547f4d4e45095e7b8b677bdd540898b76a0435f47f251a02b8a0a8f33"} Sep 09 02:35:06.355518 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:06.355126 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:06.357104 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:06.357076 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" event={"ID":"54e7834d-b125-48e4-88be-860e40a5bfdb","Type":"ContainerStarted","Data":"e831fdd83866327acb0ebb1528902d71c580d3f1c528baee544d9bd21096085d"} Sep 09 02:35:06.357230 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:06.357208 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" Sep 09 02:35:06.372031 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:06.371980 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" podStartSLOduration=2.435771666 podStartE2EDuration="4.371965363s" totalImagesPullingTime="1.936193697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:35:06.370817411 +0000 UTC m=+575.313386325" watchObservedRunningTime="2026-09-09 02:35:06.371965363 +0000 UTC m=+575.314534273" Sep 09 02:35:06.392328 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:06.392255 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" podStartSLOduration=1.423036707 podStartE2EDuration="4.392228169s" totalImagesPullingTime="2.969191462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:35:06.388083587 +0000 UTC m=+575.330652499" watchObservedRunningTime="2026-09-09 02:35:06.392228169 +0000 UTC m=+575.334797082" Sep 09 02:35:10.062752 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:10.062710 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:35:12.367690 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:12.367659 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5dfc675bbf-fcp2w" Sep 09 02:35:12.388705 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:12.388671 2577 generic.go:356] "Generic (PLEG): container finished" podID="7198324c-df6f-42f5-addd-827e1021a416" containerID="c5423d5e560ada9dcb79394e9313b414a8925e3c69f8d0360e6a2689deccf499" exitCode=0 Sep 09 02:35:12.388705 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:12.388715 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" event={"ID":"7198324c-df6f-42f5-addd-827e1021a416","Type":"ContainerDied","Data":"c5423d5e560ada9dcb79394e9313b414a8925e3c69f8d0360e6a2689deccf499"} Sep 09 02:35:12.389155 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:12.389136 2577 scope.go:117] "RemoveContainer" containerID="c5423d5e560ada9dcb79394e9313b414a8925e3c69f8d0360e6a2689deccf499" Sep 09 02:35:12.390398 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:12.390288 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 02:35:12.390398 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:12.390386 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 02:35:12.390547 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:12.390441 2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 02:35:12.901430 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:12.901383 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:35:13.399487 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:13.399438 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" event={"ID":"7198324c-df6f-42f5-addd-827e1021a416","Type":"ContainerStarted","Data":"ad7e0a64be79711823a8c38363e5e76237b335393265119d6daf9d41994ed3cd"} Sep 09 02:35:13.443346 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:13.443314 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:35:13.454324 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:13.454291 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:35:13.455392 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:13.455368 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-x8mr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Sep 09 02:35:13.455522 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:13.455413 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" podUID="7198324c-df6f-42f5-addd-827e1021a416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Sep 09 02:35:13.544619 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:13.544585 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:35:14.454865 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:14.454825 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-x8mr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Sep 09 02:35:14.455328 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:14.454894 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" podUID="7198324c-df6f-42f5-addd-827e1021a416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Sep 09 02:35:15.455081 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:15.455041 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-x8mr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Sep 09 02:35:15.455569 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:15.455111 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" podUID="7198324c-df6f-42f5-addd-827e1021a416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Sep 09 02:35:16.455204 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:16.455167 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-x8mr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Sep 09 02:35:16.455724 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:16.455223 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" podUID="7198324c-df6f-42f5-addd-827e1021a416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Sep 09 02:35:17.364252 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:17.364220 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-47qh8" Sep 09 02:35:17.454746 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:17.454715 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-x8mr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Sep 09 02:35:17.454917 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:17.454772 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" podUID="7198324c-df6f-42f5-addd-827e1021a416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Sep 09 02:35:18.454568 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:18.454533 2577 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-x8mr9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Sep 09 02:35:18.454968 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:18.454589 2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" podUID="7198324c-df6f-42f5-addd-827e1021a416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Sep 09 02:35:19.459194 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:19.459162 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:35:19.459590 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:19.459471 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:35:19.460160 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:19.460141 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-x8mr9" Sep 09 02:35:31.519723 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:31.519692 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/2.log" Sep 09 02:35:31.520140 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:31.519692 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/2.log" Sep 09 02:35:37.533561 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:35:37.533523 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:36:07.871872 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:07.871782 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:36:25.469241 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:25.469198 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:36:38.336109 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.336073 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f"] Sep 09 02:36:38.339993 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.339970 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.343272 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.343243 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 09 02:36:38.343442 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.343284 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 09 02:36:38.343442 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.343296 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tp54s\"" Sep 09 02:36:38.343442 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.343300 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 09 02:36:38.349282 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.349258 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f"] Sep 09 02:36:38.362944 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.362908 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:36:38.451370 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.451330 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9c71d48-046b-4d26-acbb-619edc5542d0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.451370 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.451376 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9c71d48-046b-4d26-acbb-619edc5542d0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.451584 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.451413 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9c71d48-046b-4d26-acbb-619edc5542d0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.451584 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.451438 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9c71d48-046b-4d26-acbb-619edc5542d0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.451584 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.451455 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np5vj\" (UniqueName: \"kubernetes.io/projected/f9c71d48-046b-4d26-acbb-619edc5542d0-kube-api-access-np5vj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.451584 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.451552 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9c71d48-046b-4d26-acbb-619edc5542d0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.553021 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.552979 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9c71d48-046b-4d26-acbb-619edc5542d0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.553209 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.553032 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9c71d48-046b-4d26-acbb-619edc5542d0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.553209 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.553059 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9c71d48-046b-4d26-acbb-619edc5542d0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.553209 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.553087 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9c71d48-046b-4d26-acbb-619edc5542d0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.553209 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.553111 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9c71d48-046b-4d26-acbb-619edc5542d0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.553209 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.553128 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np5vj\" (UniqueName: \"kubernetes.io/projected/f9c71d48-046b-4d26-acbb-619edc5542d0-kube-api-access-np5vj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.553517 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.553486 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9c71d48-046b-4d26-acbb-619edc5542d0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.553588 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.553508 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9c71d48-046b-4d26-acbb-619edc5542d0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.553588 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.553571 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9c71d48-046b-4d26-acbb-619edc5542d0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.555446 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.555423 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9c71d48-046b-4d26-acbb-619edc5542d0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.555575 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.555559 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9c71d48-046b-4d26-acbb-619edc5542d0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.560528 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.560502 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np5vj\" (UniqueName: \"kubernetes.io/projected/f9c71d48-046b-4d26-acbb-619edc5542d0-kube-api-access-np5vj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-s5h5f\" (UID: \"f9c71d48-046b-4d26-acbb-619edc5542d0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.650835 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.650740 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:38.922276 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.922161 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f"] Sep 09 02:36:38.924887 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:36:38.924859 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9c71d48_046b_4d26_acbb_619edc5542d0.slice/crio-4e59fbe3b2a84e2eebafa57a124534d186898e29fd8f56ba11b04a804f9f5cff WatchSource:0}: Error finding container 4e59fbe3b2a84e2eebafa57a124534d186898e29fd8f56ba11b04a804f9f5cff: Status 404 returned error can't find the container with id 4e59fbe3b2a84e2eebafa57a124534d186898e29fd8f56ba11b04a804f9f5cff Sep 09 02:36:38.926586 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:38.926568 2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 02:36:39.828443 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:39.828397 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" event={"ID":"f9c71d48-046b-4d26-acbb-619edc5542d0","Type":"ContainerStarted","Data":"4e59fbe3b2a84e2eebafa57a124534d186898e29fd8f56ba11b04a804f9f5cff"} Sep 09 02:36:40.838362 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:40.838321 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" event={"ID":"f9c71d48-046b-4d26-acbb-619edc5542d0","Type":"ContainerStarted","Data":"08d27362b8a640f158362da745433f2e12abc516343a2e16be3616d57209e824"} Sep 09 02:36:40.855872 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:40.855804 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" podStartSLOduration=1.86485357 podStartE2EDuration="2.855784373s" totalImagesPullingTime="990.930803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:36:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:36:40.855497493 +0000 UTC m=+669.798066402" watchObservedRunningTime="2026-09-09 02:36:40.855784373 +0000 UTC m=+669.798353284" Sep 09 02:36:48.651552 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:48.651511 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:48.651981 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:48.651603 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:48.664777 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:48.664741 2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:36:48.893765 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:36:48.893732 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-s5h5f" Sep 09 02:37:02.894794 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:37:02.894757 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:37:04.640146 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:37:04.640115 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:37:06.250800 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:37:06.250761 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:37:16.121697 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:37:16.121654 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:37:28.473624 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:37:28.473534 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:39:15.036413 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:15.036372 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d48778f78-cmnb4"] Sep 09 02:39:15.040081 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:15.040059 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d48778f78-cmnb4" Sep 09 02:39:15.051289 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:15.051257 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d48778f78-cmnb4"] Sep 09 02:39:15.172534 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:15.172495 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b0c81fb-2055-45ca-b941-6227ac6311ad-tls-cert\") pod \"authorino-6d48778f78-cmnb4\" (UID: \"2b0c81fb-2055-45ca-b941-6227ac6311ad\") " pod="kuadrant-system/authorino-6d48778f78-cmnb4" Sep 09 02:39:15.172534 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:15.172532 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqlbf\" (UniqueName: \"kubernetes.io/projected/2b0c81fb-2055-45ca-b941-6227ac6311ad-kube-api-access-kqlbf\") pod \"authorino-6d48778f78-cmnb4\" (UID: \"2b0c81fb-2055-45ca-b941-6227ac6311ad\") " pod="kuadrant-system/authorino-6d48778f78-cmnb4" Sep 09 02:39:15.273317 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:15.273274 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b0c81fb-2055-45ca-b941-6227ac6311ad-tls-cert\") pod \"authorino-6d48778f78-cmnb4\" (UID: \"2b0c81fb-2055-45ca-b941-6227ac6311ad\") " pod="kuadrant-system/authorino-6d48778f78-cmnb4" Sep 09 02:39:15.273317 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:15.273314 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqlbf\" (UniqueName: \"kubernetes.io/projected/2b0c81fb-2055-45ca-b941-6227ac6311ad-kube-api-access-kqlbf\") pod \"authorino-6d48778f78-cmnb4\" (UID: \"2b0c81fb-2055-45ca-b941-6227ac6311ad\") " pod="kuadrant-system/authorino-6d48778f78-cmnb4" Sep 09 02:39:15.275680 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:15.275653 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b0c81fb-2055-45ca-b941-6227ac6311ad-tls-cert\") pod \"authorino-6d48778f78-cmnb4\" (UID: \"2b0c81fb-2055-45ca-b941-6227ac6311ad\") " pod="kuadrant-system/authorino-6d48778f78-cmnb4" Sep 09 02:39:15.281479 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:15.281451 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqlbf\" (UniqueName: \"kubernetes.io/projected/2b0c81fb-2055-45ca-b941-6227ac6311ad-kube-api-access-kqlbf\") pod \"authorino-6d48778f78-cmnb4\" (UID: \"2b0c81fb-2055-45ca-b941-6227ac6311ad\") " pod="kuadrant-system/authorino-6d48778f78-cmnb4" Sep 09 02:39:15.349523 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:15.349421 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d48778f78-cmnb4" Sep 09 02:39:15.617951 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:15.617917 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d48778f78-cmnb4"] Sep 09 02:39:15.620198 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:39:15.620168 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b0c81fb_2055_45ca_b941_6227ac6311ad.slice/crio-da57692341360e0fddde68e219f69cdd06deba9d6fa6fc91cf7d5ffbe95258e8 WatchSource:0}: Error finding container da57692341360e0fddde68e219f69cdd06deba9d6fa6fc91cf7d5ffbe95258e8: Status 404 returned error can't find the container with id da57692341360e0fddde68e219f69cdd06deba9d6fa6fc91cf7d5ffbe95258e8 Sep 09 02:39:16.621028 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:16.620918 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d48778f78-cmnb4" event={"ID":"2b0c81fb-2055-45ca-b941-6227ac6311ad","Type":"ContainerStarted","Data":"e6150afeeeca24d3e7353ed64ec8b9851fb3c4dd52f71a822d1e0585776ae6fb"} Sep 09 02:39:16.621028 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:16.620982 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6d48778f78-cmnb4" event={"ID":"2b0c81fb-2055-45ca-b941-6227ac6311ad","Type":"ContainerStarted","Data":"da57692341360e0fddde68e219f69cdd06deba9d6fa6fc91cf7d5ffbe95258e8"} Sep 09 02:39:16.634545 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:16.634475 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6d48778f78-cmnb4" podStartSLOduration=0.957914489 podStartE2EDuration="1.634449087s" totalImagesPullingTime="676.534598ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 02:39:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:39:16.632685405 +0000 UTC m=+825.575254316" watchObservedRunningTime="2026-09-09 02:39:16.634449087 +0000 UTC m=+825.577017998" Sep 09 02:39:16.653967 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:16.653921 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67db6955c5-26qzq"] Sep 09 02:39:16.655890 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:16.655561 2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-67db6955c5-26qzq" podUID="66831307-d7a7-4cba-9d79-74dac3615bf8" containerName="authorino" containerID="cri-o://51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4" gracePeriod=30 Sep 09 02:39:16.927614 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:16.927578 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67db6955c5-26qzq" Sep 09 02:39:17.091169 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.091128 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/66831307-d7a7-4cba-9d79-74dac3615bf8-tls-cert\") pod \"66831307-d7a7-4cba-9d79-74dac3615bf8\" (UID: \"66831307-d7a7-4cba-9d79-74dac3615bf8\") " Sep 09 02:39:17.091326 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.091281 2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gnsh2\" (UniqueName: \"kubernetes.io/projected/66831307-d7a7-4cba-9d79-74dac3615bf8-kube-api-access-gnsh2\") pod \"66831307-d7a7-4cba-9d79-74dac3615bf8\" (UID: \"66831307-d7a7-4cba-9d79-74dac3615bf8\") " Sep 09 02:39:17.093297 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.093267 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66831307-d7a7-4cba-9d79-74dac3615bf8-kube-api-access-gnsh2" (OuterVolumeSpecName: "kube-api-access-gnsh2") pod "66831307-d7a7-4cba-9d79-74dac3615bf8" (UID: "66831307-d7a7-4cba-9d79-74dac3615bf8"). InnerVolumeSpecName "kube-api-access-gnsh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 02:39:17.102397 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.102367 2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66831307-d7a7-4cba-9d79-74dac3615bf8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "66831307-d7a7-4cba-9d79-74dac3615bf8" (UID: "66831307-d7a7-4cba-9d79-74dac3615bf8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 02:39:17.192532 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.192446 2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnsh2\" (UniqueName: \"kubernetes.io/projected/66831307-d7a7-4cba-9d79-74dac3615bf8-kube-api-access-gnsh2\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:39:17.192532 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.192489 2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/66831307-d7a7-4cba-9d79-74dac3615bf8-tls-cert\") on node \"ip-10-0-128-153.ec2.internal\" DevicePath \"\"" Sep 09 02:39:17.271565 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.271531 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:39:17.347371 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.347335 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:39:17.628040 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.627994 2577 generic.go:356] "Generic (PLEG): container finished" podID="66831307-d7a7-4cba-9d79-74dac3615bf8" containerID="51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4" exitCode=0 Sep 09 02:39:17.628040 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.628035 2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-67db6955c5-26qzq" Sep 09 02:39:17.628508 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.628070 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67db6955c5-26qzq" event={"ID":"66831307-d7a7-4cba-9d79-74dac3615bf8","Type":"ContainerDied","Data":"51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4"} Sep 09 02:39:17.628508 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.628126 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-67db6955c5-26qzq" event={"ID":"66831307-d7a7-4cba-9d79-74dac3615bf8","Type":"ContainerDied","Data":"ef39569136a3ba974ef52ea135ca8282d98c315b8ad513e6ff295abf9a284c3e"} Sep 09 02:39:17.628508 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.628147 2577 scope.go:117] "RemoveContainer" containerID="51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4" Sep 09 02:39:17.637328 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.637315 2577 scope.go:117] "RemoveContainer" containerID="51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4" Sep 09 02:39:17.637620 ip-10-0-128-153 kubenswrapper[2577]: E0909 02:39:17.637586 2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4\": container with ID starting with 51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4 not found: ID does not exist" containerID="51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4" Sep 09 02:39:17.637620 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.637613 2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4"} err="failed to get container status \"51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4\": rpc error: code = NotFound desc = could not find container \"51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4\": container with ID starting with 51e2740ed5ae3a7a6a9b685723848f1c5adbf9336a2d2f2926aa61dac55321f4 not found: ID does not exist" Sep 09 02:39:17.643222 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.643196 2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-67db6955c5-26qzq"] Sep 09 02:39:17.646919 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:17.646899 2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-67db6955c5-26qzq"] Sep 09 02:39:18.595757 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:18.595719 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:39:18.697067 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:18.697025 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:39:19.581812 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:19.581773 2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66831307-d7a7-4cba-9d79-74dac3615bf8" path="/var/lib/kubelet/pods/66831307-d7a7-4cba-9d79-74dac3615bf8/volumes" Sep 09 02:39:39.597362 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:39.597325 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d48778f78-cmnb4_2b0c81fb-2055-45ca-b941-6227ac6311ad/authorino/0.log" Sep 09 02:39:45.640799 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:45.640763 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w4fpd_402645c3-56ce-422a-a0d2-b27191896773/manager/0.log" Sep 09 02:39:45.777531 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:45.777466 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vl89q_712d8722-7e34-43cd-b29d-18fe0638ab1c/manager/0.log" Sep 09 02:39:45.928199 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:45.928121 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-95bf97f48-7hcbc_c2ad3753-1412-4cc5-b4b4-a4e7402c2217/manager/0.log" Sep 09 02:39:46.475965 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:46.475923 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66fb449fdb-9btn4_fe4c8400-b959-4fb0-ae5f-a6aed33fcad5/manager/0.log" Sep 09 02:39:46.481783 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:46.481729 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66fb449fdb-9btn4_fe4c8400-b959-4fb0-ae5f-a6aed33fcad5/copy-manifests/0.log" Sep 09 02:39:47.012133 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:47.012105 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5dfc675bbf-fcp2w_79c43c00-6038-4adf-8b64-899ecd155a34/maas-api/0.log" Sep 09 02:39:47.127310 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:47.127260 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-4862n_14768d61-1723-4768-b9cb-2801d3966763/postgres/0.log" Sep 09 02:39:47.985993 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:47.985932 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d48778f78-cmnb4_2b0c81fb-2055-45ca-b941-6227ac6311ad/authorino/0.log" Sep 09 02:39:48.127617 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:48.127585 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-swwcp_c22d3db0-d742-4316-87ab-0de3d06e1c00/manager/0.log" Sep 09 02:39:48.251263 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:48.251228 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-zg7pg_0c56197e-99e8-4392-bf0d-cfc0e36555e6/manager/0.log" Sep 09 02:39:48.380418 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:48.380389 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-j2c45_f46f2b07-7dce-4923-9d4c-6a2f4052640c/kuadrant-console-plugin/0.log" Sep 09 02:39:48.515649 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:48.515558 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-sbn4r_4e4c07e1-0dd9-48df-96ea-6533468d146a/manager/0.log" Sep 09 02:39:48.778729 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:48.778648 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-r69dh_7afcb052-d8ca-4a7e-9d85-0fcce7270162/manager/0.log" Sep 09 02:39:49.451290 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:49.451250 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-564645476c-8ds8p_75f273ad-48b1-47a3-8d2a-c4e777c5786a/kube-auth-proxy/0.log" Sep 09 02:39:49.716825 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:49.716747 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-x8mr9_7198324c-df6f-42f5-addd-827e1021a416/istio-proxy/1.log" Sep 09 02:39:49.963724 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:49.963693 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-47qh8_54e7834d-b125-48e4-88be-860e40a5bfdb/payload-processing/0.log" Sep 09 02:39:50.084481 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:50.084452 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b49ccfd44-hkkz2_4f03a4a6-921a-4b9c-9bba-5de2850a8b70/router/0.log" Sep 09 02:39:50.870360 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:50.870325 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-s5h5f_f9c71d48-046b-4d26-acbb-619edc5542d0/main/0.log" Sep 09 02:39:57.528849 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:57.528790 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:39:57.649352 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:39:57.649311 2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" limit=1024 Sep 09 02:40:00.519764 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:00.519736 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rczng_b87ad709-411d-4c4c-81a7-100bf4ac44d5/global-pull-secret-syncer/0.log" Sep 09 02:40:00.604346 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:00.604313 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-smfw9_953c8a38-a75f-4767-9866-68b3336a65fb/konnectivity-agent/0.log" Sep 09 02:40:00.687595 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:00.687560 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-153.ec2.internal_3788e9231500bda50f3f039c32df10fd/haproxy/0.log" Sep 09 02:40:03.992284 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:03.992221 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6d48778f78-cmnb4_2b0c81fb-2055-45ca-b941-6227ac6311ad/authorino/0.log" Sep 09 02:40:04.031856 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:04.031764 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-swwcp_c22d3db0-d742-4316-87ab-0de3d06e1c00/manager/0.log" Sep 09 02:40:04.062934 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:04.062905 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-zg7pg_0c56197e-99e8-4392-bf0d-cfc0e36555e6/manager/0.log" Sep 09 02:40:04.092140 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:04.092110 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-j2c45_f46f2b07-7dce-4923-9d4c-6a2f4052640c/kuadrant-console-plugin/0.log" Sep 09 02:40:04.139588 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:04.139556 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-sbn4r_4e4c07e1-0dd9-48df-96ea-6533468d146a/manager/0.log" Sep 09 02:40:04.197163 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:04.197126 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-r69dh_7afcb052-d8ca-4a7e-9d85-0fcce7270162/manager/0.log" Sep 09 02:40:06.746493 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:06.746456 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-xmrb5_5f61825a-519f-42c0-8ab0-3e5fcbd37b19/cluster-monitoring-operator/0.log" Sep 09 02:40:07.099949 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:07.099849 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k4wbl_8b60dbcc-a928-483e-805e-6ea855cc4959/node-exporter/0.log" Sep 09 02:40:07.117542 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:07.117514 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k4wbl_8b60dbcc-a928-483e-805e-6ea855cc4959/kube-rbac-proxy/0.log" Sep 09 02:40:07.135245 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:07.135217 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k4wbl_8b60dbcc-a928-483e-805e-6ea855cc4959/init-textfile/0.log" Sep 09 02:40:07.407935 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:07.407839 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-9v69n_19dd1e0c-909e-4ef8-8c92-46e4f98024e3/prometheus-operator/0.log" Sep 09 02:40:07.426525 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:07.426495 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-9v69n_19dd1e0c-909e-4ef8-8c92-46e4f98024e3/kube-rbac-proxy/0.log" Sep 09 02:40:07.449523 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:07.449492 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-zbmwj_1624d2d9-ca8a-46b7-ac6c-b72663f1d49c/prometheus-operator-admission-webhook/0.log" Sep 09 02:40:10.131292 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:10.131250 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/2.log" Sep 09 02:40:10.139291 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:10.139260 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-d4vlb_58860e10-d2bc-4a1a-8b98-20d772421b9f/console-operator/3.log" Sep 09 02:40:10.686390 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:10.686361 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-wrhv7_3ca0346c-d2df-4fda-9620-4b8837d64de8/download-server/0.log" Sep 09 02:40:11.240780 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.240744 2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd"] Sep 09 02:40:11.241147 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.241122 2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66831307-d7a7-4cba-9d79-74dac3615bf8" containerName="authorino" Sep 09 02:40:11.241147 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.241134 2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="66831307-d7a7-4cba-9d79-74dac3615bf8" containerName="authorino" Sep 09 02:40:11.241241 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.241230 2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="66831307-d7a7-4cba-9d79-74dac3615bf8" containerName="authorino" Sep 09 02:40:11.244471 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.244452 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.246977 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.246960 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b28xp\"/\"openshift-service-ca.crt\"" Sep 09 02:40:11.247099 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.246963 2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b28xp\"/\"default-dockercfg-7crwr\"" Sep 09 02:40:11.247099 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.246994 2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b28xp\"/\"kube-root-ca.crt\"" Sep 09 02:40:11.253876 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.253855 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd"] Sep 09 02:40:11.284004 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.283965 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/86759fa7-ccce-44db-8b94-276a710becaa-proc\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.284004 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.284009 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/86759fa7-ccce-44db-8b94-276a710becaa-lib-modules\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.284216 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.284119 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/86759fa7-ccce-44db-8b94-276a710becaa-podres\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.284216 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.284143 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86759fa7-ccce-44db-8b94-276a710becaa-sys\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.284216 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.284172 2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7m54\" (UniqueName: \"kubernetes.io/projected/86759fa7-ccce-44db-8b94-276a710becaa-kube-api-access-f7m54\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.385243 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.385202 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/86759fa7-ccce-44db-8b94-276a710becaa-podres\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.385243 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.385241 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86759fa7-ccce-44db-8b94-276a710becaa-sys\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.385464 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.385271 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7m54\" (UniqueName: \"kubernetes.io/projected/86759fa7-ccce-44db-8b94-276a710becaa-kube-api-access-f7m54\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.385464 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.385303 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/86759fa7-ccce-44db-8b94-276a710becaa-proc\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.385464 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.385346 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/86759fa7-ccce-44db-8b94-276a710becaa-sys\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.385464 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.385346 2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/86759fa7-ccce-44db-8b94-276a710becaa-lib-modules\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.385464 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.385370 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/86759fa7-ccce-44db-8b94-276a710becaa-podres\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.385464 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.385394 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/86759fa7-ccce-44db-8b94-276a710becaa-proc\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.385709 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.385528 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/86759fa7-ccce-44db-8b94-276a710becaa-lib-modules\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.394079 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.394046 2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7m54\" (UniqueName: \"kubernetes.io/projected/86759fa7-ccce-44db-8b94-276a710becaa-kube-api-access-f7m54\") pod \"perf-node-gather-daemonset-5mmwd\" (UID: \"86759fa7-ccce-44db-8b94-276a710becaa\") " pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.398932 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.398904 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-225b5_6ff5de1e-f270-439b-a4e0-3bb1394ecb79/volume-data-source-validator/0.log" Sep 09 02:40:11.554604 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.554448 2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.827172 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.827148 2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd"] Sep 09 02:40:11.829231 ip-10-0-128-153 kubenswrapper[2577]: W0909 02:40:11.829205 2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod86759fa7_ccce_44db_8b94_276a710becaa.slice/crio-c7705228ca5defe0be7f7f6923009d76f89015b0ef95f924cc2b7adedf98071d WatchSource:0}: Error finding container c7705228ca5defe0be7f7f6923009d76f89015b0ef95f924cc2b7adedf98071d: Status 404 returned error can't find the container with id c7705228ca5defe0be7f7f6923009d76f89015b0ef95f924cc2b7adedf98071d Sep 09 02:40:11.911045 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.911010 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" event={"ID":"86759fa7-ccce-44db-8b94-276a710becaa","Type":"ContainerStarted","Data":"1f464b1c75f2479919382fd159f939e78fa9a89f5f9ac10e9cda62d2914deed4"} Sep 09 02:40:11.911194 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.911065 2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" event={"ID":"86759fa7-ccce-44db-8b94-276a710becaa","Type":"ContainerStarted","Data":"c7705228ca5defe0be7f7f6923009d76f89015b0ef95f924cc2b7adedf98071d"} Sep 09 02:40:11.911194 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.911110 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:11.926387 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:11.926337 2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" podStartSLOduration=0.92632082 podStartE2EDuration="926.32082ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 02:40:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 02:40:11.924854255 +0000 UTC m=+880.867423165" watchObservedRunningTime="2026-09-09 02:40:11.92632082 +0000 UTC m=+880.868889767" Sep 09 02:40:12.569066 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:12.569038 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7nbg2_34a35235-50fe-4c00-970b-7d1d13fb59aa/dns/0.log" Sep 09 02:40:12.587892 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:12.587863 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7nbg2_34a35235-50fe-4c00-970b-7d1d13fb59aa/kube-rbac-proxy/0.log" Sep 09 02:40:12.655760 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:12.655728 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sffcl_d5d907cf-31d4-42d9-b7e1-69ee4f71c967/dns-node-resolver/0.log" Sep 09 02:40:13.202069 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:13.202031 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-t4fn5_2f661248-5853-4ec4-99ef-f03ae698507b/node-ca/0.log" Sep 09 02:40:14.205216 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:14.205173 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-564645476c-8ds8p_75f273ad-48b1-47a3-8d2a-c4e777c5786a/kube-auth-proxy/0.log" Sep 09 02:40:14.253163 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:14.253131 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-x8mr9_7198324c-df6f-42f5-addd-827e1021a416/istio-proxy/0.log" Sep 09 02:40:14.253386 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:14.253219 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-x8mr9_7198324c-df6f-42f5-addd-827e1021a416/istio-proxy/1.log" Sep 09 02:40:14.297456 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:14.297428 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-47qh8_54e7834d-b125-48e4-88be-860e40a5bfdb/payload-processing/0.log" Sep 09 02:40:14.318489 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:14.318455 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b49ccfd44-hkkz2_4f03a4a6-921a-4b9c-9bba-5de2850a8b70/router/0.log" Sep 09 02:40:14.939257 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:14.939214 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-92stt_f73d14e5-2b62-4cb1-962d-c2b4b74446f7/serve-healthcheck-canary/0.log" Sep 09 02:40:15.435889 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:15.435855 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-lh5dp_15536224-40d3-43d7-b5df-c2a44121f29d/insights-operator/0.log" Sep 09 02:40:15.436380 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:15.436217 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-lh5dp_15536224-40d3-43d7-b5df-c2a44121f29d/insights-operator/1.log" Sep 09 02:40:15.519802 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:15.519771 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l4gcf_fdbde3d2-c31b-45fb-af40-a4c2afea6265/kube-rbac-proxy/0.log" Sep 09 02:40:15.540151 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:15.540122 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l4gcf_fdbde3d2-c31b-45fb-af40-a4c2afea6265/exporter/0.log" Sep 09 02:40:15.560217 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:15.560179 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-l4gcf_fdbde3d2-c31b-45fb-af40-a4c2afea6265/extractor/0.log" Sep 09 02:40:17.689946 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:17.689912 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-w4fpd_402645c3-56ce-422a-a0d2-b27191896773/manager/0.log" Sep 09 02:40:17.730338 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:17.730288 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vl89q_712d8722-7e34-43cd-b29d-18fe0638ab1c/manager/0.log" Sep 09 02:40:17.765940 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:17.765883 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-95bf97f48-7hcbc_c2ad3753-1412-4cc5-b4b4-a4e7402c2217/manager/0.log" Sep 09 02:40:17.928126 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:17.928097 2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b28xp/perf-node-gather-daemonset-5mmwd" Sep 09 02:40:17.962884 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:17.962798 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66fb449fdb-9btn4_fe4c8400-b959-4fb0-ae5f-a6aed33fcad5/manager/0.log" Sep 09 02:40:17.977934 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:17.977912 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66fb449fdb-9btn4_fe4c8400-b959-4fb0-ae5f-a6aed33fcad5/copy-manifests/0.log" Sep 09 02:40:21.264981 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:21.264952 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-564645476c-8ds8p_75f273ad-48b1-47a3-8d2a-c4e777c5786a/kube-auth-proxy/0.log" Sep 09 02:40:21.305789 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:21.305756 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-x8mr9_7198324c-df6f-42f5-addd-827e1021a416/istio-proxy/0.log" Sep 09 02:40:21.305980 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:21.305962 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-x8mr9_7198324c-df6f-42f5-addd-827e1021a416/istio-proxy/1.log" Sep 09 02:40:21.337944 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:21.337917 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-47qh8_54e7834d-b125-48e4-88be-860e40a5bfdb/payload-processing/0.log" Sep 09 02:40:21.365116 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:21.365085 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b49ccfd44-hkkz2_4f03a4a6-921a-4b9c-9bba-5de2850a8b70/router/0.log" Sep 09 02:40:22.858665 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:22.858533 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-564645476c-8ds8p_75f273ad-48b1-47a3-8d2a-c4e777c5786a/kube-auth-proxy/0.log" Sep 09 02:40:22.894667 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:22.894625 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-x8mr9_7198324c-df6f-42f5-addd-827e1021a416/istio-proxy/0.log" Sep 09 02:40:22.894984 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:22.894962 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-x8mr9_7198324c-df6f-42f5-addd-827e1021a416/istio-proxy/1.log" Sep 09 02:40:22.926528 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:22.926504 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-47qh8_54e7834d-b125-48e4-88be-860e40a5bfdb/payload-processing/0.log" Sep 09 02:40:22.954330 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:22.954300 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b49ccfd44-hkkz2_4f03a4a6-921a-4b9c-9bba-5de2850a8b70/router/0.log" Sep 09 02:40:24.905050 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:24.905020 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-ttjnc_3018c2e4-70dc-4421-8bb0-540126c40bd5/kube-storage-version-migrator-operator/1.log" Sep 09 02:40:24.907051 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:24.907024 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-ttjnc_3018c2e4-70dc-4421-8bb0-540126c40bd5/kube-storage-version-migrator-operator/0.log" Sep 09 02:40:25.889277 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:25.889237 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4cghs_454c2cf7-2e0e-4f6a-a40e-33965988ce44/kube-multus/0.log" Sep 09 02:40:25.942507 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:25.942481 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97k2r_b6b7de4b-5150-40f8-9ef4-f6c50994bb97/kube-multus-additional-cni-plugins/0.log" Sep 09 02:40:25.963110 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:25.963084 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97k2r_b6b7de4b-5150-40f8-9ef4-f6c50994bb97/egress-router-binary-copy/0.log" Sep 09 02:40:25.982676 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:25.982617 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97k2r_b6b7de4b-5150-40f8-9ef4-f6c50994bb97/cni-plugins/0.log" Sep 09 02:40:26.001610 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:26.001582 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97k2r_b6b7de4b-5150-40f8-9ef4-f6c50994bb97/bond-cni-plugin/0.log" Sep 09 02:40:26.021778 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:26.021733 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97k2r_b6b7de4b-5150-40f8-9ef4-f6c50994bb97/routeoverride-cni/0.log" Sep 09 02:40:26.040841 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:26.040810 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97k2r_b6b7de4b-5150-40f8-9ef4-f6c50994bb97/whereabouts-cni-bincopy/0.log" Sep 09 02:40:26.059960 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:26.059936 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-97k2r_b6b7de4b-5150-40f8-9ef4-f6c50994bb97/whereabouts-cni/0.log" Sep 09 02:40:26.582953 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:26.582919 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-szmkq_06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab/network-metrics-daemon/0.log" Sep 09 02:40:26.599271 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:26.599246 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-szmkq_06f4a14b-cdf9-42b2-97a8-a0f3323cf0ab/kube-rbac-proxy/0.log" Sep 09 02:40:27.396800 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:27.396766 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78gs8_8dfc061f-fdd5-4e0d-8a05-9e3219360bd9/ovn-controller/0.log" Sep 09 02:40:27.411452 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:27.411427 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78gs8_8dfc061f-fdd5-4e0d-8a05-9e3219360bd9/ovn-acl-logging/0.log" Sep 09 02:40:27.433694 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:27.433661 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78gs8_8dfc061f-fdd5-4e0d-8a05-9e3219360bd9/kube-rbac-proxy-node/0.log" Sep 09 02:40:27.453282 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:27.453251 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78gs8_8dfc061f-fdd5-4e0d-8a05-9e3219360bd9/kube-rbac-proxy-ovn-metrics/0.log" Sep 09 02:40:27.469428 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:27.469405 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78gs8_8dfc061f-fdd5-4e0d-8a05-9e3219360bd9/northd/0.log" Sep 09 02:40:27.487981 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:27.487952 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78gs8_8dfc061f-fdd5-4e0d-8a05-9e3219360bd9/nbdb/0.log" Sep 09 02:40:27.506487 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:27.506448 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78gs8_8dfc061f-fdd5-4e0d-8a05-9e3219360bd9/sbdb/0.log" Sep 09 02:40:27.565203 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:27.565163 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-78gs8_8dfc061f-fdd5-4e0d-8a05-9e3219360bd9/ovnkube-controller/0.log" Sep 09 02:40:29.062793 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:29.062761 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-685456655f-l47l7_28b8bb88-2eef-4b7a-bd55-16549de8228f/check-endpoints/0.log" Sep 09 02:40:29.084571 ip-10-0-128-153 kubenswrapper[2577]: I0909 02:40:29.084529 2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-4w7rt_c7aaf17f-dc8e-4c1c-a4d2-5bcc64cb4016/network-check-target-container/0.log"