Sep 19 09:49:28.545398 ip-10-0-139-28 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 19 09:49:28.545417 ip-10-0-139-28 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 19 09:49:28.545426 ip-10-0-139-28 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 19 09:49:28.545643 ip-10-0-139-28 systemd[1]: Failed to start Kubernetes Kubelet. Sep 19 09:49:38.613492 ip-10-0-139-28 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 19 09:49:38.613515 ip-10-0-139-28 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c87deec6799b4fa1b9d6c49fab86a62c -- Sep 19 09:51:32.017941 ip-10-0-139-28 systemd[1]: Starting Kubernetes Kubelet... Sep 19 09:51:32.427917 ip-10-0-139-28 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 19 09:51:32.427917 ip-10-0-139-28 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 19 09:51:32.427917 ip-10-0-139-28 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 19 09:51:32.427917 ip-10-0-139-28 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 19 09:51:32.427917 ip-10-0-139-28 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 19 09:51:32.428850 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.428768 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 19 09:51:32.435783 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435754 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 09:51:32.435783 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435778 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 09:51:32.435783 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435783 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 19 09:51:32.435783 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435787 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 09:51:32.435783 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435791 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435795 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435799 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435804 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435807 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435811 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435815 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435819 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435822 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435825 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435828 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435835 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435840 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435844 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435847 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435850 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435854 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435858 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435862 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435866 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 09:51:32.436096 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435870 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435874 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435877 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435881 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435885 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435889 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435893 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435897 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435901 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435904 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435908 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435912 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435915 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435920 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435925 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435929 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435934 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435938 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435942 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435946 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 09:51:32.436949 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435950 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435954 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435958 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435962 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435966 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435970 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435975 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435978 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435983 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435987 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435991 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435995 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.435999 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436004 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436008 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436012 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436016 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436020 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436024 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436030 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 09:51:32.437791 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436034 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436038 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436043 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436047 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436051 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436055 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436060 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436067 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436073 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436077 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436082 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436086 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436090 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436098 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436105 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436109 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436114 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436119 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436123 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 09:51:32.438621 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436138 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436143 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436148 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436153 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436158 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436162 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436166 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436171 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436175 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436179 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436183 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436188 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436192 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436196 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436200 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.436204 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437464 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437474 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437480 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437485 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 09:51:32.439439 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437490 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437494 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437499 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437503 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437507 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437512 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437516 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437520 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437524 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437528 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437532 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437539 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437545 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437558 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437563 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437567 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437572 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437578 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437584 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 09:51:32.440316 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437589 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437593 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437598 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437603 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437607 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437612 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437616 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437620 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437626 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437631 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437636 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437640 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437644 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437648 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437653 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437679 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437683 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437688 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437692 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437696 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 09:51:32.440983 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437700 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437704 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437708 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437713 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437716 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437720 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437725 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437737 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437742 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437746 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437750 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437754 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437758 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437762 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437767 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437771 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437775 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437779 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437783 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437787 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 09:51:32.441558 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437790 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437794 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437799 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437803 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437807 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437811 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437815 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437822 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437828 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437833 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437840 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437844 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437849 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437853 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437857 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437861 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437865 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437869 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 09:51:32.442109 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437873 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437879 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437885 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437890 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437894 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437898 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437902 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437906 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437910 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437914 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437918 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437922 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437927 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437931 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437935 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437940 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437944 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.437948 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438597 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438605 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 09:51:32.442543 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438609 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438614 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438619 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438623 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438627 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438631 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438636 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438640 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438644 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438648 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438652 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438676 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438681 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438685 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438693 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438699 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438704 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438708 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438713 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 09:51:32.443279 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438717 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438721 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438725 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438729 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438734 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438738 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438742 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438746 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438750 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438755 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438760 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438764 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438769 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438774 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438778 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438783 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438787 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438791 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438795 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438799 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 09:51:32.443780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438803 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438807 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438812 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438816 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438822 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438826 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438830 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438834 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438840 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438844 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438848 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438852 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438856 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438861 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438865 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438868 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438873 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438877 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438881 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438884 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 09:51:32.444322 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438888 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438893 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438898 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438902 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438907 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438911 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438915 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438919 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438923 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438927 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438931 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438935 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438940 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438947 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438951 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438955 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438959 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438963 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438967 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438971 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 09:51:32.444907 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438976 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438982 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438986 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438990 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438995 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.438999 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.439003 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.439010 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.439016 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.439023 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.439028 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.439034 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.439039 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.439044 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.439049 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.439053 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.439058 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.439062 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439171 2568 flags.go:64] FLAG: --address="0.0.0.0" Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439225 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 19 09:51:32.445411 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439234 2568 flags.go:64] FLAG: --anonymous-auth="true" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439240 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439248 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439253 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439260 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439267 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439271 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439276 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439282 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439287 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439292 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439297 2568 flags.go:64] FLAG: --cgroup-root="" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439301 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439306 2568 flags.go:64] FLAG: --client-ca-file="" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439312 2568 flags.go:64] FLAG: --cloud-provider="external" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439317 2568 flags.go:64] FLAG: --cluster-dns="[]" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439324 2568 flags.go:64] FLAG: --cluster-domain="" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439328 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439333 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439338 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439344 2568 flags.go:64] FLAG: --container-log-max-files="5" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439352 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439356 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 19 09:51:32.445917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439361 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439367 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439375 2568 flags.go:64] FLAG: --contention-profiling="false" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439380 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439384 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439389 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439394 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439400 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439405 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439410 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439414 2568 flags.go:64] FLAG: --enable-load-reader="false" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439419 2568 flags.go:64] FLAG: --enable-server="true" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439424 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439430 2568 flags.go:64] FLAG: --event-burst="100" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439436 2568 flags.go:64] FLAG: --event-qps="50" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439440 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439445 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439450 2568 flags.go:64] FLAG: --eviction-hard="" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439455 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439460 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439464 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439469 2568 flags.go:64] FLAG: --eviction-soft="" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439474 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439479 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439485 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 19 09:51:32.446590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439489 2568 flags.go:64] FLAG: --experimental-mounter-path="" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439503 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439508 2568 flags.go:64] FLAG: --fail-swap-on="true" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439513 2568 flags.go:64] FLAG: --feature-gates="" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439519 2568 flags.go:64] FLAG: --file-check-frequency="20s" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439526 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439531 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439536 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439541 2568 flags.go:64] FLAG: --healthz-port="10248" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439548 2568 flags.go:64] FLAG: --help="false" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439552 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439557 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439562 2568 flags.go:64] FLAG: --http-check-frequency="20s" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439567 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439572 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439578 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439583 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439587 2568 flags.go:64] FLAG: --image-service-endpoint="" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439592 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439597 2568 flags.go:64] FLAG: --kube-api-burst="100" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439601 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439607 2568 flags.go:64] FLAG: --kube-api-qps="50" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439611 2568 flags.go:64] FLAG: --kube-reserved="" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439617 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 19 09:51:32.447230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439621 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439626 2568 flags.go:64] FLAG: --kubelet-cgroups="" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439630 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439635 2568 flags.go:64] FLAG: --lock-file="" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439639 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439644 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439648 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439677 2568 flags.go:64] FLAG: --log-json-split-stream="false" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439683 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439688 2568 flags.go:64] FLAG: --log-text-split-stream="false" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439692 2568 flags.go:64] FLAG: --logging-format="text" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439697 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439702 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439708 2568 flags.go:64] FLAG: --manifest-url="" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439713 2568 flags.go:64] FLAG: --manifest-url-header="" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439719 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439725 2568 flags.go:64] FLAG: --max-open-files="1000000" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439733 2568 flags.go:64] FLAG: --max-pods="110" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439738 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439742 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439747 2568 flags.go:64] FLAG: --memory-manager-policy="None" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439751 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439756 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439761 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439765 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 19 09:51:32.447913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439778 2568 flags.go:64] FLAG: --node-status-max-images="50" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439783 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439788 2568 flags.go:64] FLAG: --oom-score-adj="-999" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439792 2568 flags.go:64] FLAG: --pod-cidr="" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439797 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439806 2568 flags.go:64] FLAG: --pod-manifest-path="" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439810 2568 flags.go:64] FLAG: --pod-max-pids="-1" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439815 2568 flags.go:64] FLAG: --pods-per-core="0" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439820 2568 flags.go:64] FLAG: --port="10250" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439824 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439829 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08f83a31ba88fbdb4" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439834 2568 flags.go:64] FLAG: --qos-reserved="" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439838 2568 flags.go:64] FLAG: --read-only-port="10255" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439843 2568 flags.go:64] FLAG: --register-node="true" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439848 2568 flags.go:64] FLAG: --register-with-taints="" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439854 2568 flags.go:64] FLAG: --registry-burst="10" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439859 2568 flags.go:64] FLAG: --registry-qps="5" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439863 2568 flags.go:64] FLAG: --reserved-cpus="" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439867 2568 flags.go:64] FLAG: --reserved-memory="" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439874 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439880 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439885 2568 flags.go:64] FLAG: --rotate-certificates="false" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439890 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439894 2568 flags.go:64] FLAG: --runonce="false" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439899 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 19 09:51:32.448531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439905 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439910 2568 flags.go:64] FLAG: --seccomp-default="false" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439915 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439919 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439924 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439930 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439934 2568 flags.go:64] FLAG: --storage-driver-password="root" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439939 2568 flags.go:64] FLAG: --storage-driver-secure="false" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439944 2568 flags.go:64] FLAG: --storage-driver-table="stats" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439949 2568 flags.go:64] FLAG: --storage-driver-user="root" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439954 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439959 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439963 2568 flags.go:64] FLAG: --system-cgroups="" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439967 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439976 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439980 2568 flags.go:64] FLAG: --tls-cert-file="" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439985 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439992 2568 flags.go:64] FLAG: --tls-min-version="" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.439996 2568 flags.go:64] FLAG: --tls-private-key-file="" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.440001 2568 flags.go:64] FLAG: --topology-manager-policy="none" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.440006 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.440011 2568 flags.go:64] FLAG: --topology-manager-scope="container" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.440016 2568 flags.go:64] FLAG: --v="2" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.440022 2568 flags.go:64] FLAG: --version="false" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.440029 2568 flags.go:64] FLAG: --vmodule="" Sep 19 09:51:32.449163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.440035 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.440040 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440234 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440241 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440247 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440251 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440255 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440261 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440268 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440274 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440279 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440284 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440289 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440294 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440298 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440303 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440307 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440312 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440316 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 09:51:32.449811 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440320 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440324 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440328 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440332 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440336 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440340 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440344 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440349 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440353 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440357 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440362 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440366 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440370 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440375 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440379 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440385 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440389 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440393 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440398 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440402 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 09:51:32.450277 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440407 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440411 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440416 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440420 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440424 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440429 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440433 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440437 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440441 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440447 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440453 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440457 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440461 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440466 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440470 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440475 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440479 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440483 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440487 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 09:51:32.450780 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440491 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440495 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440499 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440508 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440512 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440517 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440521 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440526 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440533 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440537 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440541 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440545 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440549 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440556 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440560 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440564 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440568 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440573 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440577 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440581 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 09:51:32.451244 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440585 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440589 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440593 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440597 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440601 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440605 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440609 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440614 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440618 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440622 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440626 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440630 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440634 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440638 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440643 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440647 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440651 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440675 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440682 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440687 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 09:51:32.451847 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440695 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440700 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.440704 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.441292 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.448965 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.448982 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.450372 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.450379 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450449 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450454 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450457 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450460 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450463 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450466 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450468 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 09:51:32.452327 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450471 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450474 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450477 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450481 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450485 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450488 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450491 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450494 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450497 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450500 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450502 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450505 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450508 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450511 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450513 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450516 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450518 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450522 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450524 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450527 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 09:51:32.452718 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450530 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450533 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450536 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450538 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450541 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450544 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450546 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450549 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450551 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450554 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450557 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450559 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450562 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450564 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450567 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450570 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450572 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450575 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450577 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450580 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 09:51:32.453238 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450582 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450585 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450587 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450591 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450593 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450596 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450598 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450601 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450603 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450606 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450609 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450612 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450615 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450617 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450620 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450623 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450626 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450629 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450631 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450634 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 09:51:32.453745 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450636 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450639 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450641 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450646 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450649 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450652 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450674 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450678 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450680 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450683 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450685 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450688 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450690 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450694 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450696 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450698 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450701 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450704 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450706 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 09:51:32.454216 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450709 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450713 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450717 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450719 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450723 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450726 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450728 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450732 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450735 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450737 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450740 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450742 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450745 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.450750 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450864 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 19 09:51:32.454681 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450870 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450873 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450876 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450879 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450882 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450885 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450888 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450890 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450893 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450896 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450899 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450901 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450904 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450907 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450909 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450912 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450915 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450918 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450920 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450923 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 19 09:51:32.455052 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450925 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450928 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450931 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450933 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450936 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450939 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450941 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450943 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450946 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450949 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450951 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450954 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450956 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450959 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450963 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450966 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450969 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450971 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450974 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 19 09:51:32.455552 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450977 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450980 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450982 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450984 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450987 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450991 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450994 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450997 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.450999 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451002 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451005 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451007 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451010 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451014 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451016 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451019 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451022 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451025 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451027 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451030 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 19 09:51:32.456034 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451032 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451034 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451037 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451039 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451042 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451044 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451047 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451049 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451051 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451054 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451057 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451059 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451062 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451064 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451067 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451088 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451091 2568 feature_gate.go:328] unrecognized feature gate: Example2 Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451094 2568 feature_gate.go:328] unrecognized feature gate: Example Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451097 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451100 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 19 09:51:32.456500 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451103 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451106 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451108 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451111 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451113 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451116 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451118 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451121 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451123 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451127 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451129 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451132 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451134 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451137 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451140 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451142 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451145 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451147 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 19 09:51:32.456995 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:32.451150 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 19 09:51:32.457418 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.451155 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 19 09:51:32.457418 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.451275 2568 server.go:962] "Client rotation is on, will bootstrap in background" Sep 19 09:51:32.457418 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.453334 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 19 09:51:32.457418 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.454294 2568 server.go:1019] "Starting client certificate rotation" Sep 19 09:51:32.457418 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.454397 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 19 09:51:32.457418 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.454429 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 19 09:51:32.457418 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.457365 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 19 09:51:32.462817 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.462788 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 19 09:51:32.476426 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.476402 2568 log.go:25] "Validated CRI v1 runtime API" Sep 19 09:51:32.478111 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.478090 2568 log.go:25] "Validated CRI v1 image API" Sep 19 09:51:32.478994 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.478979 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 19 09:51:32.482392 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.482371 2568 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7e42302a-61e4-4c7d-9e36-a876f4826bb3:/dev/nvme0n1p4 ce3e75d1-6076-4240-871e-e06d2c6a59d8:/dev/nvme0n1p3] Sep 19 09:51:32.482466 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.482389 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 19 09:51:32.484057 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.484039 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 19 09:51:32.487498 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.487378 2568 manager.go:216] Machine: {Timestamp:2026-09-19 09:51:32.486090214 +0000 UTC m=+0.349079964 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091431 MemoryCapacity:32812150784 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29dde2c8e7f3dff9630adb69e78071 SystemUUID:ec29dde2-c8e7-f3df-f963-0adb69e78071 BootID:c87deec6-799b-4fa1-b9d6-c49fab86a62c Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406073344 Type:vfs Inodes:4005389 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ea:0c:52:bb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ea:0c:52:bb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:e3:c8:3e:66:72 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812150784 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 19 09:51:32.487498 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.487488 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 19 09:51:32.487622 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.487609 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 19 09:51:32.490930 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.490906 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 19 09:51:32.491078 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.490934 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-28.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 19 09:51:32.491810 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.491800 2568 topology_manager.go:138] "Creating topology manager with none policy" Sep 19 09:51:32.491850 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.491814 2568 container_manager_linux.go:309] "Creating device plugin manager" Sep 19 09:51:32.491850 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.491822 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 19 09:51:32.492584 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.492574 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 19 09:51:32.493896 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.493886 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 19 09:51:32.497917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.497906 2568 state_mem.go:36] "Initialized new in-memory state store" Sep 19 09:51:32.498037 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.498028 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 19 09:51:32.498084 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.498056 2568 kubelet.go:486] "Attempting to sync node with API server" Sep 19 09:51:32.498084 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.498066 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 19 09:51:32.498084 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.498080 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 19 09:51:32.498207 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.498090 2568 kubelet.go:398] "Adding apiserver pod source" Sep 19 09:51:32.498207 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.498161 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 19 09:51:32.498553 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.498540 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 19 09:51:32.498606 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.498561 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 19 09:51:32.501080 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.501060 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1" Sep 19 09:51:32.503571 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.503557 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 19 09:51:32.503637 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.503588 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 19 09:51:32.504690 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.504677 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 19 09:51:32.504738 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.504694 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 19 09:51:32.504738 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.504700 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 19 09:51:32.504738 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.504705 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 19 09:51:32.504738 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.504713 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 19 09:51:32.504738 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.504719 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 19 09:51:32.504738 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.504731 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 19 09:51:32.504738 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.504737 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 19 09:51:32.504947 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.504743 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 19 09:51:32.504947 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.504749 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 19 09:51:32.504947 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.504758 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 19 09:51:32.504947 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.504763 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 19 09:51:32.505554 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.505543 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 19 09:51:32.505586 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.505555 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 19 09:51:32.506560 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.506523 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bx6zs" Sep 19 09:51:32.509347 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.509120 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-28.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 19 09:51:32.509565 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.509547 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-28.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 19 09:51:32.509740 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.509715 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 19 09:51:32.509933 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.509919 2568 server.go:1268] "Started kubelet" Sep 19 09:51:32.510675 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.510630 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 19 09:51:32.510869 ip-10-0-139-28 systemd[1]: Started Kubernetes Kubelet. Sep 19 09:51:32.511111 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.510836 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 19 09:51:32.511111 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.510944 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 19 09:51:32.512372 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.512149 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 19 09:51:32.513417 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.513400 2568 server.go:310] "Adding debug handlers to kubelet server" Sep 19 09:51:32.514373 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.514357 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bx6zs" Sep 19 09:51:32.516568 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.516540 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 19 09:51:32.520926 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.519879 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-28.ec2.internal.18d6b011d1f69b19 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-28.ec2.internal,UID:ip-10-0-139-28.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-28.ec2.internal,},FirstTimestamp:2026-09-19 09:51:32.509883161 +0000 UTC m=+0.372872908,LastTimestamp:2026-09-19 09:51:32.509883161 +0000 UTC m=+0.372872908,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-28.ec2.internal,}" Sep 19 09:51:32.523122 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.523095 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 19 09:51:32.523122 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.523105 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 19 09:51:32.523285 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.523263 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:32.523847 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.523815 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 19 09:51:32.523847 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.523832 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 19 09:51:32.523847 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.523847 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 19 09:51:32.524124 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.523964 2568 reconstruct.go:97] "Volume reconstruction finished" Sep 19 09:51:32.524124 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.523972 2568 reconciler.go:29] "Reconciler: start to sync state" Sep 19 09:51:32.524219 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.524210 2568 factory.go:55] Registering systemd factory Sep 19 09:51:32.524273 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.524222 2568 factory.go:223] Registration of the systemd container factory successfully Sep 19 09:51:32.524469 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.524450 2568 factory.go:153] Registering CRI-O factory Sep 19 09:51:32.524469 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.524472 2568 factory.go:223] Registration of the crio container factory successfully Sep 19 09:51:32.524611 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.524525 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 19 09:51:32.524611 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.524544 2568 factory.go:103] Registering Raw factory Sep 19 09:51:32.524611 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.524561 2568 manager.go:1196] Started watching for new ooms in manager Sep 19 09:51:32.525037 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.525020 2568 manager.go:349] Starting recovery of all containers Sep 19 09:51:32.533991 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.533964 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:51:32.534920 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.534905 2568 manager.go:354] Recovery completed Sep 19 09:51:32.537318 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.537300 2568 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-28.ec2.internal\" not found" node="ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.538875 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.538862 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:51:32.540901 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.540886 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:51:32.540967 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.540920 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:51:32.540967 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.540930 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:51:32.541473 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.541461 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 19 09:51:32.541533 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.541474 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 19 09:51:32.541533 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.541492 2568 state_mem.go:36] "Initialized new in-memory state store" Sep 19 09:51:32.541604 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.541533 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:51:32.543376 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.543363 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:51:32.543450 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.543395 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:51:32.543450 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.543405 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:51:32.543624 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.543612 2568 policy_none.go:49] "None policy: Start" Sep 19 09:51:32.543678 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.543631 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 19 09:51:32.543716 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.543678 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 19 09:51:32.545591 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.545580 2568 policy_none.go:47] "Start" Sep 19 09:51:32.582265 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.582247 2568 manager.go:337] "Starting Device Plugin manager" Sep 19 09:51:32.607600 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.582289 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 19 09:51:32.607600 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.582302 2568 server.go:87] "Starting device plugin registration server" Sep 19 09:51:32.607600 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.582565 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 19 09:51:32.607600 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.582577 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 19 09:51:32.607600 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.582682 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 19 09:51:32.607600 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.582770 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 19 09:51:32.607600 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.582779 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 19 09:51:32.607600 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.583336 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 19 09:51:32.607600 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.583388 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:32.622717 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.622681 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 19 09:51:32.624065 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.624042 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 19 09:51:32.624163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.624103 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 19 09:51:32.624163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.624132 2568 kubelet.go:2448] "Starting kubelet main sync loop" Sep 19 09:51:32.624250 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.624177 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 19 09:51:32.626434 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.626418 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:51:32.683566 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.683434 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:51:32.684470 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.684447 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:51:32.684579 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.684482 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:51:32.684579 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.684493 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:51:32.684579 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.684523 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.690297 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.690280 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.690376 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.690306 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-28.ec2.internal\": node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:32.704230 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.704212 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:32.725250 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.725225 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal"] Sep 19 09:51:32.725318 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.725287 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:51:32.726787 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.726772 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:51:32.726857 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.726802 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:51:32.726857 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.726818 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:51:32.728835 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.728823 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:51:32.728967 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.728951 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.729005 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.728985 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:51:32.729600 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.729584 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:51:32.729600 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.729595 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:51:32.729724 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.729612 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:51:32.729724 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.729615 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:51:32.729724 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.729622 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:51:32.729724 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.729624 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:51:32.731691 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.731652 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.731760 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.731712 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 19 09:51:32.732448 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.732435 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientMemory" Sep 19 09:51:32.732512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.732459 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasNoDiskPressure" Sep 19 09:51:32.732512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.732469 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeHasSufficientPID" Sep 19 09:51:32.749284 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.749253 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-28.ec2.internal\" not found" node="ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.753835 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.753816 2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-28.ec2.internal\" not found" node="ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.804559 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.804524 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:32.825125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.825100 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fcbba51457286c61d2f15a5467b9e8e5-config\") pod \"kube-apiserver-proxy-ip-10-0-139-28.ec2.internal\" (UID: \"fcbba51457286c61d2f15a5467b9e8e5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.825214 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.825130 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c905b7f78aba76fc624a8ba36df86b52-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal\" (UID: \"c905b7f78aba76fc624a8ba36df86b52\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.825214 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.825149 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c905b7f78aba76fc624a8ba36df86b52-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal\" (UID: \"c905b7f78aba76fc624a8ba36df86b52\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.904939 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:32.904896 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:32.926300 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.926277 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fcbba51457286c61d2f15a5467b9e8e5-config\") pod \"kube-apiserver-proxy-ip-10-0-139-28.ec2.internal\" (UID: \"fcbba51457286c61d2f15a5467b9e8e5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.926373 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.926307 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c905b7f78aba76fc624a8ba36df86b52-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal\" (UID: \"c905b7f78aba76fc624a8ba36df86b52\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.926373 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.926327 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c905b7f78aba76fc624a8ba36df86b52-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal\" (UID: \"c905b7f78aba76fc624a8ba36df86b52\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.926465 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.926391 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c905b7f78aba76fc624a8ba36df86b52-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal\" (UID: \"c905b7f78aba76fc624a8ba36df86b52\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.926465 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.926404 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/c905b7f78aba76fc624a8ba36df86b52-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal\" (UID: \"c905b7f78aba76fc624a8ba36df86b52\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Sep 19 09:51:32.926465 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:32.926425 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fcbba51457286c61d2f15a5467b9e8e5-config\") pod \"kube-apiserver-proxy-ip-10-0-139-28.ec2.internal\" (UID: \"fcbba51457286c61d2f15a5467b9e8e5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" Sep 19 09:51:33.005766 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:33.005691 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:33.053269 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.053243 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" Sep 19 09:51:33.057381 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.057361 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Sep 19 09:51:33.106151 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:33.106113 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:33.206719 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:33.206674 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:33.307281 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:33.307213 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:33.407750 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:33.407712 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:33.435580 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.435548 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:51:33.453820 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.453783 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 19 09:51:33.453985 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.453966 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 19 09:51:33.454040 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.453988 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 19 09:51:33.454040 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.453988 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 19 09:51:33.508324 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:33.508280 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:33.520503 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.520456 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 09:46:33 +0000 UTC" deadline="2028-04-28 10:18:38.631587461 +0000 UTC" Sep 19 09:51:33.520503 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.520498 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14088h27m5.111092881s" Sep 19 09:51:33.609454 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:33.609259 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-28.ec2.internal\" not found" Sep 19 09:51:33.630347 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.630324 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:51:33.681626 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:33.681595 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc905b7f78aba76fc624a8ba36df86b52.slice/crio-8990248ed0709dcdd5472bcd7113cbf1a050823297c2eb67197db21e214232a0 WatchSource:0}: Error finding container 8990248ed0709dcdd5472bcd7113cbf1a050823297c2eb67197db21e214232a0: Status 404 returned error can't find the container with id 8990248ed0709dcdd5472bcd7113cbf1a050823297c2eb67197db21e214232a0 Sep 19 09:51:33.682405 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:33.682379 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcbba51457286c61d2f15a5467b9e8e5.slice/crio-01a93f9fa62a35294ec9bde5bc524689771e49760c5979c1506733ff399f7e72 WatchSource:0}: Error finding container 01a93f9fa62a35294ec9bde5bc524689771e49760c5979c1506733ff399f7e72: Status 404 returned error can't find the container with id 01a93f9fa62a35294ec9bde5bc524689771e49760c5979c1506733ff399f7e72 Sep 19 09:51:33.687235 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.687219 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 09:51:33.724526 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.724499 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" Sep 19 09:51:33.735602 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.735580 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 19 09:51:33.736503 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.736488 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" Sep 19 09:51:33.745931 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:33.745909 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 19 09:51:35.955115 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.955065 2568 apiserver.go:52] "Watching apiserver" Sep 19 09:51:35.961467 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.961437 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:51:35.962179 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.962158 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 19 09:51:35.964572 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.964549 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-pzsvj","kube-system/konnectivity-agent-79pp9","kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal","openshift-cluster-node-tuning-operator/tuned-q9lm4","openshift-multus/multus-additional-cni-plugins-bxkqq","openshift-multus/multus-pllt7","openshift-multus/network-metrics-daemon-hrrnt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77","openshift-dns/node-resolver-jddfm","openshift-image-registry/node-ca-24d99","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal","openshift-network-diagnostics/network-check-target-kl7xh","openshift-network-operator/iptables-alerter-rlb2k","openshift-ovn-kubernetes/ovnkube-node-55lq9"] Sep 19 09:51:35.967190 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.967171 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:35.967281 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:35.967259 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:35.967937 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.967914 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:51:35.969397 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.969378 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rlb2k" Sep 19 09:51:35.971890 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.971866 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 19 09:51:35.972152 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.972117 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-psbkj\"" Sep 19 09:51:35.972311 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.972297 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 19 09:51:35.972485 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.972472 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 19 09:51:35.972763 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.972749 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.974615 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.974360 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.975018 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.975001 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pp45x\"" Sep 19 09:51:35.975772 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.975746 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 19 09:51:35.975866 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.975815 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 19 09:51:35.975866 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.975851 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 19 09:51:35.976008 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.975981 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 19 09:51:35.976106 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.976012 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 19 09:51:35.976106 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.976055 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 19 09:51:35.976494 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.976474 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 19 09:51:35.976887 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.976724 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:35.976887 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.976781 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 19 09:51:35.979280 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.978833 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8p9xx\"" Sep 19 09:51:35.979280 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.978857 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 19 09:51:35.979280 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.979022 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hcnlm\"" Sep 19 09:51:35.979280 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.979041 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 19 09:51:35.979280 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.979230 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 19 09:51:35.979470 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.979325 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.979470 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.979383 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 19 09:51:35.979470 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.979330 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 19 09:51:35.979729 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.979714 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 19 09:51:35.982786 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.982758 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-v5mwg\"" Sep 19 09:51:35.983120 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.983033 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 19 09:51:35.985897 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.984918 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jddfm" Sep 19 09:51:35.986032 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.986013 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:35.986152 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:35.986109 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:51:35.987336 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.987297 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 19 09:51:35.987792 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.987777 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 19 09:51:35.987956 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.987777 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bnhwd\"" Sep 19 09:51:35.988580 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.988561 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-79pp9" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990092 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-var-lib-openvswitch\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990148 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8f492656-f7cf-4308-ab65-fe7b5f59c31b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990176 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-run-netns\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990200 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-hostroot\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990221 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-run-netns\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990242 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-lib-modules\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990263 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-run-ovn-kubernetes\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990286 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-sysconfig\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990328 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-run-ovn\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990353 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-cni-netd\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990375 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-run\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990413 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-multus-socket-dir-parent\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990448 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a0c22579-3d34-441c-afc8-7200ae008c73-host-slash\") pod \"iptables-alerter-rlb2k\" (UID: \"a0c22579-3d34-441c-afc8-7200ae008c73\") " pod="openshift-network-operator/iptables-alerter-rlb2k" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990469 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-node-log\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990490 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac5b2015-06bf-4a21-9879-965fc0529731-ovnkube-script-lib\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990514 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc429\" (UniqueName: \"kubernetes.io/projected/ac5b2015-06bf-4a21-9879-965fc0529731-kube-api-access-hc429\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990544 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-etc-openvswitch\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990579 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-cni-bin\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990610 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990611 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f492656-f7cf-4308-ab65-fe7b5f59c31b-system-cni-dir\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990635 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-run-systemd\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990715 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0f909b1-2fcf-4d89-ae30-67eea76335d7-tmp\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990752 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-cnibin\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990774 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-etc-kubernetes\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990799 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhhph\" (UniqueName: \"kubernetes.io/projected/a0c22579-3d34-441c-afc8-7200ae008c73-kube-api-access-qhhph\") pod \"iptables-alerter-rlb2k\" (UID: \"a0c22579-3d34-441c-afc8-7200ae008c73\") " pod="openshift-network-operator/iptables-alerter-rlb2k" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.990860 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-x5l6n\"" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991059 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991119 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8f492656-f7cf-4308-ab65-fe7b5f59c31b-cni-binary-copy\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991171 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-tuned\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991237 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-run-multus-certs\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991379 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991405 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-systemd-units\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991429 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-slash\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.991815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991473 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-run-openvswitch\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991498 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b3d2ddbc-feb3-4edb-9713-5d1822cce2b8-hosts-file\") pod \"node-resolver-jddfm\" (UID: \"b3d2ddbc-feb3-4edb-9713-5d1822cce2b8\") " pod="openshift-dns/node-resolver-jddfm" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991598 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-var-lib-kubelet\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991625 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-run-k8s-cni-cncf-io\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991661 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-var-lib-cni-multus\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-var-lib-kubelet\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991723 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-modprobe-d\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991749 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-host\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991773 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-multus-conf-dir\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991786 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-24d99" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991821 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991846 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8f492656-f7cf-4308-ab65-fe7b5f59c31b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991869 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a0c22579-3d34-441c-afc8-7200ae008c73-iptables-alerter-script\") pod \"iptables-alerter-rlb2k\" (UID: \"a0c22579-3d34-441c-afc8-7200ae008c73\") " pod="openshift-network-operator/iptables-alerter-rlb2k" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991894 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac5b2015-06bf-4a21-9879-965fc0529731-env-overrides\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.991979 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-sysctl-conf\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992017 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-system-cni-dir\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992040 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rljl\" (UniqueName: \"kubernetes.io/projected/6bd97875-f073-455a-bbaa-17fa3c598cbe-kube-api-access-8rljl\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.992437 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992064 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8f492656-f7cf-4308-ab65-fe7b5f59c31b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992089 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxgfh\" (UniqueName: \"kubernetes.io/projected/b3d2ddbc-feb3-4edb-9713-5d1822cce2b8-kube-api-access-wxgfh\") pod \"node-resolver-jddfm\" (UID: \"b3d2ddbc-feb3-4edb-9713-5d1822cce2b8\") " pod="openshift-dns/node-resolver-jddfm" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992117 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-sysctl-d\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992201 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n488p\" (UniqueName: \"kubernetes.io/projected/e0f909b1-2fcf-4d89-ae30-67eea76335d7-kube-api-access-n488p\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992225 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-systemd\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992367 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-sys\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992390 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-kubelet\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992413 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-log-socket\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992456 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8f492656-f7cf-4308-ab65-fe7b5f59c31b-cnibin\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992480 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t77zc\" (UniqueName: \"kubernetes.io/projected/8f492656-f7cf-4308-ab65-fe7b5f59c31b-kube-api-access-t77zc\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992503 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3d2ddbc-feb3-4edb-9713-5d1822cce2b8-tmp-dir\") pod \"node-resolver-jddfm\" (UID: \"b3d2ddbc-feb3-4edb-9713-5d1822cce2b8\") " pod="openshift-dns/node-resolver-jddfm" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992525 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-kubernetes\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-multus-cni-dir\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992572 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6bd97875-f073-455a-bbaa-17fa3c598cbe-cni-binary-copy\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992595 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6bd97875-f073-455a-bbaa-17fa3c598cbe-multus-daemon-config\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992621 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-var-lib-cni-bin\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992643 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac5b2015-06bf-4a21-9879-965fc0529731-ovnkube-config\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.993125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992761 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac5b2015-06bf-4a21-9879-965fc0529731-ovn-node-metrics-cert\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:35.993722 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.992791 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5894t\" (UniqueName: \"kubernetes.io/projected/0559c450-7285-4470-9fb8-12716eff996a-kube-api-access-5894t\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:35.993786 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.993770 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:35.993945 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.993926 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 19 09:51:35.994419 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.994330 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-fr2wr\"" Sep 19 09:51:35.995022 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.994595 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 19 09:51:35.995022 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.994761 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 19 09:51:35.995022 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.994761 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 19 09:51:35.996044 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.996026 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 19 09:51:35.996383 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.996326 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nhwcg\"" Sep 19 09:51:35.996531 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.996517 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 19 09:51:35.996651 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.996630 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 19 09:51:35.996865 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:35.996848 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:35.996933 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:35.996908 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:51:36.014451 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.014429 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gxs7g" Sep 19 09:51:36.022046 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.022026 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gxs7g" Sep 19 09:51:36.081198 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.081164 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 19 09:51:36.084630 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.084579 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" event={"ID":"fcbba51457286c61d2f15a5467b9e8e5","Type":"ContainerStarted","Data":"01a93f9fa62a35294ec9bde5bc524689771e49760c5979c1506733ff399f7e72"} Sep 19 09:51:36.085825 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.085798 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" event={"ID":"c905b7f78aba76fc624a8ba36df86b52","Type":"ContainerStarted","Data":"8990248ed0709dcdd5472bcd7113cbf1a050823297c2eb67197db21e214232a0"} Sep 19 09:51:36.093474 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093448 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5894t\" (UniqueName: \"kubernetes.io/projected/0559c450-7285-4470-9fb8-12716eff996a-kube-api-access-5894t\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:36.093598 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093481 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-var-lib-openvswitch\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.093598 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093502 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8f492656-f7cf-4308-ab65-fe7b5f59c31b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.093598 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093530 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4dad002c-07a5-481d-8f21-b3f4ad57a2c0-konnectivity-ca\") pod \"konnectivity-agent-79pp9\" (UID: \"4dad002c-07a5-481d-8f21-b3f4ad57a2c0\") " pod="kube-system/konnectivity-agent-79pp9" Sep 19 09:51:36.093598 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093553 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-run-netns\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.093598 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093575 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-hostroot\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.093598 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093586 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-var-lib-openvswitch\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.093598 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093596 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-run-netns\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093620 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-device-dir\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093640 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-run-netns\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093643 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-lib-modules\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093672 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-run-ovn-kubernetes\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093690 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4dad002c-07a5-481d-8f21-b3f4ad57a2c0-agent-certs\") pod \"konnectivity-agent-79pp9\" (UID: \"4dad002c-07a5-481d-8f21-b3f4ad57a2c0\") " pod="kube-system/konnectivity-agent-79pp9" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093708 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-sysconfig\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093727 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-run-ovn\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093742 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-cni-netd\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093756 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-registration-dir\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093772 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-etc-selinux\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093787 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-run-netns\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093793 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-run\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093811 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-multus-socket-dir-parent\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093835 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-hostroot\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093845 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a0c22579-3d34-441c-afc8-7200ae008c73-host-slash\") pod \"iptables-alerter-rlb2k\" (UID: \"a0c22579-3d34-441c-afc8-7200ae008c73\") " pod="openshift-network-operator/iptables-alerter-rlb2k" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093841 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-lib-modules\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.093973 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093895 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-run-ovn-kubernetes\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093910 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-multus-socket-dir-parent\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093929 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-run\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093930 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-sysconfig\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093950 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-run-ovn\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093950 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-node-log\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093963 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a0c22579-3d34-441c-afc8-7200ae008c73-host-slash\") pod \"iptables-alerter-rlb2k\" (UID: \"a0c22579-3d34-441c-afc8-7200ae008c73\") " pod="openshift-network-operator/iptables-alerter-rlb2k" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093973 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-cni-netd\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.093988 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-node-log\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094000 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac5b2015-06bf-4a21-9879-965fc0529731-ovnkube-script-lib\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094016 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc429\" (UniqueName: \"kubernetes.io/projected/ac5b2015-06bf-4a21-9879-965fc0529731-kube-api-access-hc429\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094044 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-etc-openvswitch\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094080 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-cni-bin\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094107 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-cni-bin\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094116 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8f492656-f7cf-4308-ab65-fe7b5f59c31b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094116 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f492656-f7cf-4308-ab65-fe7b5f59c31b-system-cni-dir\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094121 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-etc-openvswitch\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.094827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094177 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f492656-f7cf-4308-ab65-fe7b5f59c31b-system-cni-dir\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094180 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75b47a48-c5ec-4502-b447-6c6dccf8b622-host\") pod \"node-ca-24d99\" (UID: \"75b47a48-c5ec-4502-b447-6c6dccf8b622\") " pod="openshift-image-registry/node-ca-24d99" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094222 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-run-systemd\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094257 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0f909b1-2fcf-4d89-ae30-67eea76335d7-tmp\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094287 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-cnibin\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094308 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-run-systemd\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-etc-kubernetes\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094346 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhhph\" (UniqueName: \"kubernetes.io/projected/a0c22579-3d34-441c-afc8-7200ae008c73-kube-api-access-qhhph\") pod \"iptables-alerter-rlb2k\" (UID: \"a0c22579-3d34-441c-afc8-7200ae008c73\") " pod="openshift-network-operator/iptables-alerter-rlb2k" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094353 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-etc-kubernetes\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094372 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8f492656-f7cf-4308-ab65-fe7b5f59c31b-cni-binary-copy\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094399 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094400 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-cnibin\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094433 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-sys-fs\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094474 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-tuned\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094499 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-run-multus-certs\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094522 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094567 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-systemd-units\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094595 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-slash\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.095659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094647 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-run-openvswitch\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094676 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b3d2ddbc-feb3-4edb-9713-5d1822cce2b8-hosts-file\") pod \"node-resolver-jddfm\" (UID: \"b3d2ddbc-feb3-4edb-9713-5d1822cce2b8\") " pod="openshift-dns/node-resolver-jddfm" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094686 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac5b2015-06bf-4a21-9879-965fc0529731-ovnkube-script-lib\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094704 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3875e600-bb82-4e1a-8e93-3afae30494f7-kubelet-config\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094732 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-var-lib-kubelet\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094755 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-run-k8s-cni-cncf-io\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094815 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-var-lib-cni-multus\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094843 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-run-multus-certs\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094857 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-var-lib-kubelet\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094869 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094885 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-socket-dir\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094890 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-systemd-units\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094911 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-modprobe-d\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094920 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-slash\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094937 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-host\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094946 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-run-openvswitch\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-multus-conf-dir\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.094989 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b3d2ddbc-feb3-4edb-9713-5d1822cce2b8-hosts-file\") pod \"node-resolver-jddfm\" (UID: \"b3d2ddbc-feb3-4edb-9713-5d1822cce2b8\") " pod="openshift-dns/node-resolver-jddfm" Sep 19 09:51:36.097442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095027 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-run-k8s-cni-cncf-io\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095051 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-var-lib-kubelet\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095060 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-var-lib-cni-multus\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095081 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-var-lib-kubelet\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095092 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8f492656-f7cf-4308-ab65-fe7b5f59c31b-cni-binary-copy\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095111 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095160 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8f492656-f7cf-4308-ab65-fe7b5f59c31b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095166 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-multus-conf-dir\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095172 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-modprobe-d\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095188 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/75b47a48-c5ec-4502-b447-6c6dccf8b622-serviceca\") pod \"node-ca-24d99\" (UID: \"75b47a48-c5ec-4502-b447-6c6dccf8b622\") " pod="openshift-image-registry/node-ca-24d99" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095212 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.095245 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095248 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-host\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095306 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8f492656-f7cf-4308-ab65-fe7b5f59c31b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.095330 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs podName:0559c450-7285-4470-9fb8-12716eff996a nodeName:}" failed. No retries permitted until 2026-09-19 09:51:36.595301425 +0000 UTC m=+3.002813075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs") pod "network-metrics-daemon-hrrnt" (UID: "0559c450-7285-4470-9fb8-12716eff996a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095353 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk69k\" (UniqueName: \"kubernetes.io/projected/75b47a48-c5ec-4502-b447-6c6dccf8b622-kube-api-access-bk69k\") pod \"node-ca-24d99\" (UID: \"75b47a48-c5ec-4502-b447-6c6dccf8b622\") " pod="openshift-image-registry/node-ca-24d99" Sep 19 09:51:36.098294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095385 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk5sk\" (UniqueName: \"kubernetes.io/projected/223993d8-8704-4f44-a51b-89729f591c1a-kube-api-access-kk5sk\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095423 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a0c22579-3d34-441c-afc8-7200ae008c73-iptables-alerter-script\") pod \"iptables-alerter-rlb2k\" (UID: \"a0c22579-3d34-441c-afc8-7200ae008c73\") " pod="openshift-network-operator/iptables-alerter-rlb2k" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095443 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac5b2015-06bf-4a21-9879-965fc0529731-env-overrides\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095486 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n96bf\" (UniqueName: \"kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf\") pod \"network-check-target-kl7xh\" (UID: \"99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e\") " pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095516 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-sysctl-conf\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095539 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-system-cni-dir\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095576 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rljl\" (UniqueName: \"kubernetes.io/projected/6bd97875-f073-455a-bbaa-17fa3c598cbe-kube-api-access-8rljl\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095662 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8f492656-f7cf-4308-ab65-fe7b5f59c31b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095689 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxgfh\" (UniqueName: \"kubernetes.io/projected/b3d2ddbc-feb3-4edb-9713-5d1822cce2b8-kube-api-access-wxgfh\") pod \"node-resolver-jddfm\" (UID: \"b3d2ddbc-feb3-4edb-9713-5d1822cce2b8\") " pod="openshift-dns/node-resolver-jddfm" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095696 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-system-cni-dir\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095714 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-sysctl-d\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095736 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n488p\" (UniqueName: \"kubernetes.io/projected/e0f909b1-2fcf-4d89-ae30-67eea76335d7-kube-api-access-n488p\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095757 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-systemd\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095777 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-sys\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095796 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-kubelet\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095815 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-log-socket\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095835 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8f492656-f7cf-4308-ab65-fe7b5f59c31b-cnibin\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.099076 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095856 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t77zc\" (UniqueName: \"kubernetes.io/projected/8f492656-f7cf-4308-ab65-fe7b5f59c31b-kube-api-access-t77zc\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095862 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-sysctl-d\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095867 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac5b2015-06bf-4a21-9879-965fc0529731-env-overrides\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095880 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3d2ddbc-feb3-4edb-9713-5d1822cce2b8-tmp-dir\") pod \"node-resolver-jddfm\" (UID: \"b3d2ddbc-feb3-4edb-9713-5d1822cce2b8\") " pod="openshift-dns/node-resolver-jddfm" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095901 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-kubernetes\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-multus-cni-dir\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095925 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-host-kubelet\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095937 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-systemd\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095942 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6bd97875-f073-455a-bbaa-17fa3c598cbe-cni-binary-copy\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095980 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-sys\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096058 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-kubernetes\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.095879 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-sysctl-conf\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096084 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6bd97875-f073-455a-bbaa-17fa3c598cbe-multus-daemon-config\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096122 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8f492656-f7cf-4308-ab65-fe7b5f59c31b-cnibin\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096165 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3875e600-bb82-4e1a-8e93-3afae30494f7-dbus\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096162 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8f492656-f7cf-4308-ab65-fe7b5f59c31b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096194 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096188 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac5b2015-06bf-4a21-9879-965fc0529731-log-socket\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.099642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096211 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-multus-cni-dir\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.100255 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096225 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-var-lib-cni-bin\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.100255 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096250 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac5b2015-06bf-4a21-9879-965fc0529731-ovnkube-config\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.100255 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096273 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac5b2015-06bf-4a21-9879-965fc0529731-ovn-node-metrics-cert\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.100255 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096316 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3d2ddbc-feb3-4edb-9713-5d1822cce2b8-tmp-dir\") pod \"node-resolver-jddfm\" (UID: \"b3d2ddbc-feb3-4edb-9713-5d1822cce2b8\") " pod="openshift-dns/node-resolver-jddfm" Sep 19 09:51:36.100255 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096371 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6bd97875-f073-455a-bbaa-17fa3c598cbe-host-var-lib-cni-bin\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.100255 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096544 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a0c22579-3d34-441c-afc8-7200ae008c73-iptables-alerter-script\") pod \"iptables-alerter-rlb2k\" (UID: \"a0c22579-3d34-441c-afc8-7200ae008c73\") " pod="openshift-network-operator/iptables-alerter-rlb2k" Sep 19 09:51:36.100255 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096547 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6bd97875-f073-455a-bbaa-17fa3c598cbe-cni-binary-copy\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.100255 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096679 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6bd97875-f073-455a-bbaa-17fa3c598cbe-multus-daemon-config\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.100255 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.096807 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac5b2015-06bf-4a21-9879-965fc0529731-ovnkube-config\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.100255 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.099527 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0f909b1-2fcf-4d89-ae30-67eea76335d7-tmp\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.100255 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.099570 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac5b2015-06bf-4a21-9879-965fc0529731-ovn-node-metrics-cert\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.100255 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.099697 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e0f909b1-2fcf-4d89-ae30-67eea76335d7-etc-tuned\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.102413 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.101958 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhhph\" (UniqueName: \"kubernetes.io/projected/a0c22579-3d34-441c-afc8-7200ae008c73-kube-api-access-qhhph\") pod \"iptables-alerter-rlb2k\" (UID: \"a0c22579-3d34-441c-afc8-7200ae008c73\") " pod="openshift-network-operator/iptables-alerter-rlb2k" Sep 19 09:51:36.102413 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.102379 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5894t\" (UniqueName: \"kubernetes.io/projected/0559c450-7285-4470-9fb8-12716eff996a-kube-api-access-5894t\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:36.102666 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.102638 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc429\" (UniqueName: \"kubernetes.io/projected/ac5b2015-06bf-4a21-9879-965fc0529731-kube-api-access-hc429\") pod \"ovnkube-node-55lq9\" (UID: \"ac5b2015-06bf-4a21-9879-965fc0529731\") " pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.103759 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.103693 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rljl\" (UniqueName: \"kubernetes.io/projected/6bd97875-f073-455a-bbaa-17fa3c598cbe-kube-api-access-8rljl\") pod \"multus-pllt7\" (UID: \"6bd97875-f073-455a-bbaa-17fa3c598cbe\") " pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.105165 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.104070 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t77zc\" (UniqueName: \"kubernetes.io/projected/8f492656-f7cf-4308-ab65-fe7b5f59c31b-kube-api-access-t77zc\") pod \"multus-additional-cni-plugins-bxkqq\" (UID: \"8f492656-f7cf-4308-ab65-fe7b5f59c31b\") " pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.105165 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.104923 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n488p\" (UniqueName: \"kubernetes.io/projected/e0f909b1-2fcf-4d89-ae30-67eea76335d7-kube-api-access-n488p\") pod \"tuned-q9lm4\" (UID: \"e0f909b1-2fcf-4d89-ae30-67eea76335d7\") " pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.105679 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.105664 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxgfh\" (UniqueName: \"kubernetes.io/projected/b3d2ddbc-feb3-4edb-9713-5d1822cce2b8-kube-api-access-wxgfh\") pod \"node-resolver-jddfm\" (UID: \"b3d2ddbc-feb3-4edb-9713-5d1822cce2b8\") " pod="openshift-dns/node-resolver-jddfm" Sep 19 09:51:36.197598 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197562 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-registration-dir\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.197598 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197598 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-etc-selinux\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.197843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197622 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75b47a48-c5ec-4502-b447-6c6dccf8b622-host\") pod \"node-ca-24d99\" (UID: \"75b47a48-c5ec-4502-b447-6c6dccf8b622\") " pod="openshift-image-registry/node-ca-24d99" Sep 19 09:51:36.197843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197684 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75b47a48-c5ec-4502-b447-6c6dccf8b622-host\") pod \"node-ca-24d99\" (UID: \"75b47a48-c5ec-4502-b447-6c6dccf8b622\") " pod="openshift-image-registry/node-ca-24d99" Sep 19 09:51:36.197843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197703 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-registration-dir\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.197843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197710 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:36.197843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197743 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-etc-selinux\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.197843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197759 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-sys-fs\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.197843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197794 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3875e600-bb82-4e1a-8e93-3afae30494f7-kubelet-config\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:36.197843 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.197812 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:51:36.198196 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.197872 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret podName:3875e600-bb82-4e1a-8e93-3afae30494f7 nodeName:}" failed. No retries permitted until 2026-09-19 09:51:36.697852601 +0000 UTC m=+3.105364248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret") pod "global-pull-secret-syncer-pzsvj" (UID: "3875e600-bb82-4e1a-8e93-3afae30494f7") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:51:36.198196 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197873 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3875e600-bb82-4e1a-8e93-3afae30494f7-kubelet-config\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:36.198196 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197815 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-socket-dir\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.198196 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197871 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-sys-fs\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.198196 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197906 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/75b47a48-c5ec-4502-b447-6c6dccf8b622-serviceca\") pod \"node-ca-24d99\" (UID: \"75b47a48-c5ec-4502-b447-6c6dccf8b622\") " pod="openshift-image-registry/node-ca-24d99" Sep 19 09:51:36.198196 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197927 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-socket-dir\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.198196 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197931 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk69k\" (UniqueName: \"kubernetes.io/projected/75b47a48-c5ec-4502-b447-6c6dccf8b622-kube-api-access-bk69k\") pod \"node-ca-24d99\" (UID: \"75b47a48-c5ec-4502-b447-6c6dccf8b622\") " pod="openshift-image-registry/node-ca-24d99" Sep 19 09:51:36.198196 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.197976 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk5sk\" (UniqueName: \"kubernetes.io/projected/223993d8-8704-4f44-a51b-89729f591c1a-kube-api-access-kk5sk\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.198196 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.198002 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n96bf\" (UniqueName: \"kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf\") pod \"network-check-target-kl7xh\" (UID: \"99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e\") " pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:36.198196 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.198037 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3875e600-bb82-4e1a-8e93-3afae30494f7-dbus\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:36.198196 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.198058 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.198196 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.198112 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.198629 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.198227 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4dad002c-07a5-481d-8f21-b3f4ad57a2c0-konnectivity-ca\") pod \"konnectivity-agent-79pp9\" (UID: \"4dad002c-07a5-481d-8f21-b3f4ad57a2c0\") " pod="kube-system/konnectivity-agent-79pp9" Sep 19 09:51:36.198629 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.198229 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3875e600-bb82-4e1a-8e93-3afae30494f7-dbus\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:36.198629 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.198259 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-device-dir\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.198629 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.198305 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/223993d8-8704-4f44-a51b-89729f591c1a-device-dir\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.198629 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.198321 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4dad002c-07a5-481d-8f21-b3f4ad57a2c0-agent-certs\") pod \"konnectivity-agent-79pp9\" (UID: \"4dad002c-07a5-481d-8f21-b3f4ad57a2c0\") " pod="kube-system/konnectivity-agent-79pp9" Sep 19 09:51:36.198629 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.198406 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/75b47a48-c5ec-4502-b447-6c6dccf8b622-serviceca\") pod \"node-ca-24d99\" (UID: \"75b47a48-c5ec-4502-b447-6c6dccf8b622\") " pod="openshift-image-registry/node-ca-24d99" Sep 19 09:51:36.198822 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.198682 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4dad002c-07a5-481d-8f21-b3f4ad57a2c0-konnectivity-ca\") pod \"konnectivity-agent-79pp9\" (UID: \"4dad002c-07a5-481d-8f21-b3f4ad57a2c0\") " pod="kube-system/konnectivity-agent-79pp9" Sep 19 09:51:36.200746 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.200724 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4dad002c-07a5-481d-8f21-b3f4ad57a2c0-agent-certs\") pod \"konnectivity-agent-79pp9\" (UID: \"4dad002c-07a5-481d-8f21-b3f4ad57a2c0\") " pod="kube-system/konnectivity-agent-79pp9" Sep 19 09:51:36.204852 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.204830 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:51:36.204852 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.204861 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:51:36.205036 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.204877 2568 projected.go:196] Error preparing data for projected volume kube-api-access-n96bf for pod openshift-network-diagnostics/network-check-target-kl7xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:51:36.205036 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.204948 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf podName:99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e nodeName:}" failed. No retries permitted until 2026-09-19 09:51:36.704929463 +0000 UTC m=+3.112441114 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n96bf" (UniqueName: "kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf") pod "network-check-target-kl7xh" (UID: "99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:51:36.206791 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.206729 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk5sk\" (UniqueName: \"kubernetes.io/projected/223993d8-8704-4f44-a51b-89729f591c1a-kube-api-access-kk5sk\") pod \"aws-ebs-csi-driver-node-2hk77\" (UID: \"223993d8-8704-4f44-a51b-89729f591c1a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.206883 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.206794 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk69k\" (UniqueName: \"kubernetes.io/projected/75b47a48-c5ec-4502-b447-6c6dccf8b622-kube-api-access-bk69k\") pod \"node-ca-24d99\" (UID: \"75b47a48-c5ec-4502-b447-6c6dccf8b622\") " pod="openshift-image-registry/node-ca-24d99" Sep 19 09:51:36.287176 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.287126 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rlb2k" Sep 19 09:51:36.294448 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.294426 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:51:36.303918 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.303893 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" Sep 19 09:51:36.308819 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.308802 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bxkqq" Sep 19 09:51:36.315518 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.315495 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-pllt7" Sep 19 09:51:36.322050 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.322031 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jddfm" Sep 19 09:51:36.328083 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.328055 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-79pp9" Sep 19 09:51:36.334639 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.334621 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-24d99" Sep 19 09:51:36.340237 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.340216 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" Sep 19 09:51:36.566323 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.566255 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 19 09:51:36.600668 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.600637 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:36.600832 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.600765 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:51:36.600832 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.600821 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs podName:0559c450-7285-4470-9fb8-12716eff996a nodeName:}" failed. No retries permitted until 2026-09-19 09:51:37.600805325 +0000 UTC m=+4.008316974 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs") pod "network-metrics-daemon-hrrnt" (UID: "0559c450-7285-4470-9fb8-12716eff996a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:51:36.700941 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.700908 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:36.701116 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.701092 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:51:36.701199 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.701191 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret podName:3875e600-bb82-4e1a-8e93-3afae30494f7 nodeName:}" failed. No retries permitted until 2026-09-19 09:51:37.701174531 +0000 UTC m=+4.108686177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret") pod "global-pull-secret-syncer-pzsvj" (UID: "3875e600-bb82-4e1a-8e93-3afae30494f7") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:51:36.802121 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:36.802084 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n96bf\" (UniqueName: \"kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf\") pod \"network-check-target-kl7xh\" (UID: \"99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e\") " pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:36.802295 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.802271 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:51:36.802350 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.802310 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:51:36.802350 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.802327 2568 projected.go:196] Error preparing data for projected volume kube-api-access-n96bf for pod openshift-network-diagnostics/network-check-target-kl7xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:51:36.802423 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:36.802396 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf podName:99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e nodeName:}" failed. No retries permitted until 2026-09-19 09:51:37.802375231 +0000 UTC m=+4.209886898 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-n96bf" (UniqueName: "kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf") pod "network-check-target-kl7xh" (UID: "99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:51:36.970646 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:36.970622 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd97875_f073_455a_bbaa_17fa3c598cbe.slice/crio-103677e3df511ed9147f264076186a2a2db21456c067eb215213b1edce01aa81 WatchSource:0}: Error finding container 103677e3df511ed9147f264076186a2a2db21456c067eb215213b1edce01aa81: Status 404 returned error can't find the container with id 103677e3df511ed9147f264076186a2a2db21456c067eb215213b1edce01aa81 Sep 19 09:51:36.972053 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:36.972037 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dad002c_07a5_481d_8f21_b3f4ad57a2c0.slice/crio-84c4c9c39ffcda5a20aa04787aa7336e09233db56d32618c59891b34ca7788bc WatchSource:0}: Error finding container 84c4c9c39ffcda5a20aa04787aa7336e09233db56d32618c59891b34ca7788bc: Status 404 returned error can't find the container with id 84c4c9c39ffcda5a20aa04787aa7336e09233db56d32618c59891b34ca7788bc Sep 19 09:51:36.972963 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:36.972941 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5b2015_06bf_4a21_9879_965fc0529731.slice/crio-5606533b27bec19f59243102fcfba7edf026266bb3b408507bbae09c040f7632 WatchSource:0}: Error finding container 5606533b27bec19f59243102fcfba7edf026266bb3b408507bbae09c040f7632: Status 404 returned error can't find the container with id 5606533b27bec19f59243102fcfba7edf026266bb3b408507bbae09c040f7632 Sep 19 09:51:36.974326 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:36.974288 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3d2ddbc_feb3_4edb_9713_5d1822cce2b8.slice/crio-f7eab389539c7f8fd227e8d67215df72319237baca95747dcd42e87d8b9b2242 WatchSource:0}: Error finding container f7eab389539c7f8fd227e8d67215df72319237baca95747dcd42e87d8b9b2242: Status 404 returned error can't find the container with id f7eab389539c7f8fd227e8d67215df72319237baca95747dcd42e87d8b9b2242 Sep 19 09:51:36.995992 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:36.995967 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b47a48_c5ec_4502_b447_6c6dccf8b622.slice/crio-c3f9b1ee25d07ae8d4cdcf3e504ad1020db20c379a320f8a336b2da0ba838cbf WatchSource:0}: Error finding container c3f9b1ee25d07ae8d4cdcf3e504ad1020db20c379a320f8a336b2da0ba838cbf: Status 404 returned error can't find the container with id c3f9b1ee25d07ae8d4cdcf3e504ad1020db20c379a320f8a336b2da0ba838cbf Sep 19 09:51:36.996607 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:36.996572 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod223993d8_8704_4f44_a51b_89729f591c1a.slice/crio-4b86936f67e7ecfde6975647e9615200af86e987debe71e88f61922e94ab29f2 WatchSource:0}: Error finding container 4b86936f67e7ecfde6975647e9615200af86e987debe71e88f61922e94ab29f2: Status 404 returned error can't find the container with id 4b86936f67e7ecfde6975647e9615200af86e987debe71e88f61922e94ab29f2 Sep 19 09:51:36.997582 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:36.997559 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f492656_f7cf_4308_ab65_fe7b5f59c31b.slice/crio-c315f2ea35ce608dda46a2b5106db830aa01cbd6e18aac3570061ab5cffef6d8 WatchSource:0}: Error finding container c315f2ea35ce608dda46a2b5106db830aa01cbd6e18aac3570061ab5cffef6d8: Status 404 returned error can't find the container with id c315f2ea35ce608dda46a2b5106db830aa01cbd6e18aac3570061ab5cffef6d8 Sep 19 09:51:36.998750 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:36.998726 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c22579_3d34_441c_afc8_7200ae008c73.slice/crio-3a18dae5f8015a4610e286b61dcbada249e8715352538a10df0c8767bb3e43d4 WatchSource:0}: Error finding container 3a18dae5f8015a4610e286b61dcbada249e8715352538a10df0c8767bb3e43d4: Status 404 returned error can't find the container with id 3a18dae5f8015a4610e286b61dcbada249e8715352538a10df0c8767bb3e43d4 Sep 19 09:51:37.001648 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:51:37.001151 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f909b1_2fcf_4d89_ae30_67eea76335d7.slice/crio-561bfecd442a2e1825492611c4335f7690b48af5338d0bc44439fb2c907201d8 WatchSource:0}: Error finding container 561bfecd442a2e1825492611c4335f7690b48af5338d0bc44439fb2c907201d8: Status 404 returned error can't find the container with id 561bfecd442a2e1825492611c4335f7690b48af5338d0bc44439fb2c907201d8 Sep 19 09:51:37.025105 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.025062 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 09:46:36 +0000 UTC" deadline="2028-04-19 06:54:59.688386049 +0000 UTC" Sep 19 09:51:37.025105 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.025104 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13869h3m22.663285629s" Sep 19 09:51:37.080259 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.080235 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:37.080389 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:37.080369 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:37.090557 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.090528 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" event={"ID":"e0f909b1-2fcf-4d89-ae30-67eea76335d7","Type":"ContainerStarted","Data":"561bfecd442a2e1825492611c4335f7690b48af5338d0bc44439fb2c907201d8"} Sep 19 09:51:37.093262 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.093230 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" event={"ID":"223993d8-8704-4f44-a51b-89729f591c1a","Type":"ContainerStarted","Data":"4b86936f67e7ecfde6975647e9615200af86e987debe71e88f61922e94ab29f2"} Sep 19 09:51:37.094532 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.094504 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jddfm" event={"ID":"b3d2ddbc-feb3-4edb-9713-5d1822cce2b8","Type":"ContainerStarted","Data":"f7eab389539c7f8fd227e8d67215df72319237baca95747dcd42e87d8b9b2242"} Sep 19 09:51:37.095899 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.095877 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" event={"ID":"ac5b2015-06bf-4a21-9879-965fc0529731","Type":"ContainerStarted","Data":"5606533b27bec19f59243102fcfba7edf026266bb3b408507bbae09c040f7632"} Sep 19 09:51:37.096996 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.096975 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pllt7" event={"ID":"6bd97875-f073-455a-bbaa-17fa3c598cbe","Type":"ContainerStarted","Data":"103677e3df511ed9147f264076186a2a2db21456c067eb215213b1edce01aa81"} Sep 19 09:51:37.103420 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.103389 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" event={"ID":"fcbba51457286c61d2f15a5467b9e8e5","Type":"ContainerStarted","Data":"3fec27584a5eda3c6430f37334577dea0f2d279d263f7e98b1395c96a1648e5d"} Sep 19 09:51:37.106407 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.106377 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rlb2k" event={"ID":"a0c22579-3d34-441c-afc8-7200ae008c73","Type":"ContainerStarted","Data":"3a18dae5f8015a4610e286b61dcbada249e8715352538a10df0c8767bb3e43d4"} Sep 19 09:51:37.107595 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.107576 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bxkqq" event={"ID":"8f492656-f7cf-4308-ab65-fe7b5f59c31b","Type":"ContainerStarted","Data":"c315f2ea35ce608dda46a2b5106db830aa01cbd6e18aac3570061ab5cffef6d8"} Sep 19 09:51:37.108643 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.108623 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-24d99" event={"ID":"75b47a48-c5ec-4502-b447-6c6dccf8b622","Type":"ContainerStarted","Data":"c3f9b1ee25d07ae8d4cdcf3e504ad1020db20c379a320f8a336b2da0ba838cbf"} Sep 19 09:51:37.112384 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.112364 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-79pp9" event={"ID":"4dad002c-07a5-481d-8f21-b3f4ad57a2c0","Type":"ContainerStarted","Data":"84c4c9c39ffcda5a20aa04787aa7336e09233db56d32618c59891b34ca7788bc"} Sep 19 09:51:37.118055 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.118016 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-28.ec2.internal" podStartSLOduration=2.118004281 podStartE2EDuration="2.118004281s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:51:37.117974559 +0000 UTC m=+3.525486228" watchObservedRunningTime="2026-09-19 09:51:37.118004281 +0000 UTC m=+3.525515949" Sep 19 09:51:37.606898 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.606797 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:37.607065 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:37.607003 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:51:37.607128 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:37.607067 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs podName:0559c450-7285-4470-9fb8-12716eff996a nodeName:}" failed. No retries permitted until 2026-09-19 09:51:39.607048923 +0000 UTC m=+6.014560575 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs") pod "network-metrics-daemon-hrrnt" (UID: "0559c450-7285-4470-9fb8-12716eff996a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:51:37.708115 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.708069 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:37.708313 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:37.708209 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:51:37.708313 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:37.708278 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret podName:3875e600-bb82-4e1a-8e93-3afae30494f7 nodeName:}" failed. No retries permitted until 2026-09-19 09:51:39.708257775 +0000 UTC m=+6.115769443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret") pod "global-pull-secret-syncer-pzsvj" (UID: "3875e600-bb82-4e1a-8e93-3afae30494f7") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:51:37.809309 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:37.809263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n96bf\" (UniqueName: \"kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf\") pod \"network-check-target-kl7xh\" (UID: \"99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e\") " pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:37.809488 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:37.809451 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:51:37.809488 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:37.809481 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:51:37.809600 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:37.809499 2568 projected.go:196] Error preparing data for projected volume kube-api-access-n96bf for pod openshift-network-diagnostics/network-check-target-kl7xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:51:37.809600 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:37.809565 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf podName:99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e nodeName:}" failed. No retries permitted until 2026-09-19 09:51:39.809548279 +0000 UTC m=+6.217059932 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-n96bf" (UniqueName: "kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf") pod "network-check-target-kl7xh" (UID: "99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:51:38.026048 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:38.025958 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 09:46:36 +0000 UTC" deadline="2028-04-27 16:08:51.682041622 +0000 UTC" Sep 19 09:51:38.026048 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:38.025992 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14070h17m13.656054618s" Sep 19 09:51:38.082129 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:38.082097 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:38.082293 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:38.082251 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:51:38.082637 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:38.082615 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:38.082742 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:38.082709 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:51:38.130998 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:38.130951 2568 generic.go:356] "Generic (PLEG): container finished" podID="c905b7f78aba76fc624a8ba36df86b52" containerID="d574b6a14096ceff10d47043d437f8116c27383ca1b1bd9524faac022fbf1c39" exitCode=0 Sep 19 09:51:38.131261 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:38.131239 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" event={"ID":"c905b7f78aba76fc624a8ba36df86b52","Type":"ContainerDied","Data":"d574b6a14096ceff10d47043d437f8116c27383ca1b1bd9524faac022fbf1c39"} Sep 19 09:51:39.081107 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:39.080860 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:39.081564 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:39.081244 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:39.139537 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:39.138876 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" event={"ID":"c905b7f78aba76fc624a8ba36df86b52","Type":"ContainerStarted","Data":"7f0c5d99e210daae8096b21388f763bb472e40994d0c7dd94b9265bf21e399f9"} Sep 19 09:51:39.630647 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:39.630609 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:39.630816 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:39.630779 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:51:39.630892 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:39.630845 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs podName:0559c450-7285-4470-9fb8-12716eff996a nodeName:}" failed. No retries permitted until 2026-09-19 09:51:43.630826101 +0000 UTC m=+10.038337754 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs") pod "network-metrics-daemon-hrrnt" (UID: "0559c450-7285-4470-9fb8-12716eff996a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:51:39.731662 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:39.731623 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:39.731901 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:39.731881 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:51:39.731994 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:39.731963 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret podName:3875e600-bb82-4e1a-8e93-3afae30494f7 nodeName:}" failed. No retries permitted until 2026-09-19 09:51:43.731941569 +0000 UTC m=+10.139453221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret") pod "global-pull-secret-syncer-pzsvj" (UID: "3875e600-bb82-4e1a-8e93-3afae30494f7") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:51:39.832737 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:39.832694 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n96bf\" (UniqueName: \"kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf\") pod \"network-check-target-kl7xh\" (UID: \"99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e\") " pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:39.832953 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:39.832926 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:51:39.833044 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:39.832957 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:51:39.833044 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:39.832970 2568 projected.go:196] Error preparing data for projected volume kube-api-access-n96bf for pod openshift-network-diagnostics/network-check-target-kl7xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:51:39.833044 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:39.833033 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf podName:99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e nodeName:}" failed. No retries permitted until 2026-09-19 09:51:43.833011207 +0000 UTC m=+10.240522877 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-n96bf" (UniqueName: "kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf") pod "network-check-target-kl7xh" (UID: "99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:51:40.080810 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:40.080733 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:40.080985 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:40.080872 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:51:40.081356 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:40.081327 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:40.081740 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:40.081424 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:51:41.079895 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:41.079867 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:41.080065 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:41.079969 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:42.081270 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:42.081237 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:42.081725 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:42.081424 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:51:42.081808 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:42.081786 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:42.081912 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:42.081885 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:51:43.080889 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:43.080854 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:43.081085 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:43.080999 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:43.665757 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:43.665629 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:43.666125 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:43.665797 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:51:43.666125 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:43.665879 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs podName:0559c450-7285-4470-9fb8-12716eff996a nodeName:}" failed. No retries permitted until 2026-09-19 09:51:51.665848017 +0000 UTC m=+18.073359680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs") pod "network-metrics-daemon-hrrnt" (UID: "0559c450-7285-4470-9fb8-12716eff996a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:51:43.766930 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:43.766889 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:43.767117 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:43.767053 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:51:43.767117 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:43.767116 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret podName:3875e600-bb82-4e1a-8e93-3afae30494f7 nodeName:}" failed. No retries permitted until 2026-09-19 09:51:51.767098505 +0000 UTC m=+18.174610172 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret") pod "global-pull-secret-syncer-pzsvj" (UID: "3875e600-bb82-4e1a-8e93-3afae30494f7") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:51:43.868362 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:43.868324 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n96bf\" (UniqueName: \"kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf\") pod \"network-check-target-kl7xh\" (UID: \"99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e\") " pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:43.868540 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:43.868492 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:51:43.868540 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:43.868512 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:51:43.868540 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:43.868524 2568 projected.go:196] Error preparing data for projected volume kube-api-access-n96bf for pod openshift-network-diagnostics/network-check-target-kl7xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:51:43.868712 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:43.868592 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf podName:99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e nodeName:}" failed. No retries permitted until 2026-09-19 09:51:51.868573002 +0000 UTC m=+18.276084654 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-n96bf" (UniqueName: "kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf") pod "network-check-target-kl7xh" (UID: "99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:51:44.082278 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:44.082193 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:44.082435 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:44.082288 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:44.082435 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:44.082308 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:51:44.082435 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:44.082401 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:51:45.080161 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:45.080071 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:45.081066 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:45.080751 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:46.080698 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:46.080669 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:46.081177 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:46.080670 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:46.081177 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:46.080791 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:51:46.081177 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:46.080865 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:51:47.080355 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:47.080311 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:47.080518 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:47.080435 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:48.080481 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:48.080404 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:48.080481 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:48.080453 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:48.080923 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:48.080546 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:51:48.080923 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:48.080668 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:51:49.080834 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:49.080793 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:49.081293 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:49.080949 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:50.080773 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:50.080742 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:50.080936 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:50.080787 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:50.080936 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:50.080876 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:51:50.081270 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:50.081011 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:51:51.080845 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:51.080808 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:51.081075 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:51.080940 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:51.727881 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:51.727850 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:51.728030 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:51.727970 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:51:51.728030 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:51.728022 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs podName:0559c450-7285-4470-9fb8-12716eff996a nodeName:}" failed. No retries permitted until 2026-09-19 09:52:07.728007893 +0000 UTC m=+34.135519540 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs") pod "network-metrics-daemon-hrrnt" (UID: "0559c450-7285-4470-9fb8-12716eff996a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:51:51.829308 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:51.829265 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:51.829500 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:51.829444 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:51:51.829548 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:51.829524 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret podName:3875e600-bb82-4e1a-8e93-3afae30494f7 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:07.829503582 +0000 UTC m=+34.237015246 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret") pod "global-pull-secret-syncer-pzsvj" (UID: "3875e600-bb82-4e1a-8e93-3afae30494f7") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:51:51.929995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:51.929964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n96bf\" (UniqueName: \"kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf\") pod \"network-check-target-kl7xh\" (UID: \"99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e\") " pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:51.930193 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:51.930166 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:51:51.930193 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:51.930187 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:51:51.930275 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:51.930199 2568 projected.go:196] Error preparing data for projected volume kube-api-access-n96bf for pod openshift-network-diagnostics/network-check-target-kl7xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:51:51.930275 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:51.930261 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf podName:99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e nodeName:}" failed. No retries permitted until 2026-09-19 09:52:07.930244376 +0000 UTC m=+34.337756023 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-n96bf" (UniqueName: "kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf") pod "network-check-target-kl7xh" (UID: "99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:51:52.080472 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:52.080381 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:52.080638 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:52.080511 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:51:52.080638 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:52.080555 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:52.080738 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:52.080653 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:51:53.080370 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:53.080336 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:53.080788 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:53.080478 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:54.081106 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:54.081065 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:54.081572 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:54.081248 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:51:54.081572 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:54.081296 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:54.081572 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:54.081383 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:51:55.080435 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:55.080402 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:55.080584 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:55.080508 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:56.080426 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.080027 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:56.081330 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.080068 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:56.081330 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:56.080567 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:51:56.081330 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:56.080535 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:51:56.177403 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.177343 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-24d99" event={"ID":"75b47a48-c5ec-4502-b447-6c6dccf8b622","Type":"ContainerStarted","Data":"ec22abaf5476a7aec2c834d53fef66f908660eb84eb7e45c69596bde262e9fb3"} Sep 19 09:51:56.179459 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.179430 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-79pp9" event={"ID":"4dad002c-07a5-481d-8f21-b3f4ad57a2c0","Type":"ContainerStarted","Data":"1a808cecdc2bc13db8dd0c2c35a673cadf760de93f32bf64a20834c3accf4f9a"} Sep 19 09:51:56.181504 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.181092 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" event={"ID":"e0f909b1-2fcf-4d89-ae30-67eea76335d7","Type":"ContainerStarted","Data":"206a929539215728bdc8c097c21e774ba26179f32fdbd4906bc1112a21136d3f"} Sep 19 09:51:56.183197 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.183166 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" event={"ID":"223993d8-8704-4f44-a51b-89729f591c1a","Type":"ContainerStarted","Data":"9a01cbedde6e737530d95eeb33331c1de462ac166b7744988d79e44ab0f39b96"} Sep 19 09:51:56.184765 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.184732 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jddfm" event={"ID":"b3d2ddbc-feb3-4edb-9713-5d1822cce2b8","Type":"ContainerStarted","Data":"6c14a4102f03d0cccd2ae86a2b1e71c3e47a153e12826378f4af1b7b1d88be37"} Sep 19 09:51:56.187811 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.187785 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" event={"ID":"ac5b2015-06bf-4a21-9879-965fc0529731","Type":"ContainerStarted","Data":"b4a662e5d1d0e29ad233596714691c6c12ac97054133c23cd85fd763e6aee0a8"} Sep 19 09:51:56.187909 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.187870 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" event={"ID":"ac5b2015-06bf-4a21-9879-965fc0529731","Type":"ContainerStarted","Data":"27e68dc1f3f8b88090b726c3fda5627c9152986b44d1c93afa17dfdfb392be48"} Sep 19 09:51:56.187949 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.187920 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" event={"ID":"ac5b2015-06bf-4a21-9879-965fc0529731","Type":"ContainerStarted","Data":"69fa8ce558e784ac194e5656e778519729ff1e5e8cbdcb8fa0976c47bb2fc45f"} Sep 19 09:51:56.187997 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.187966 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" event={"ID":"ac5b2015-06bf-4a21-9879-965fc0529731","Type":"ContainerStarted","Data":"75a69bc2bfc2bd0f29b4a5b56ee6880580dcb929722922745ed69c43cac538c0"} Sep 19 09:51:56.188036 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.188014 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" event={"ID":"ac5b2015-06bf-4a21-9879-965fc0529731","Type":"ContainerStarted","Data":"ac62f2d36c43c67a79be6bbe22eb34c42480eeb61806b892b1d15ae99a4e00fc"} Sep 19 09:51:56.188079 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.188063 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" event={"ID":"ac5b2015-06bf-4a21-9879-965fc0529731","Type":"ContainerStarted","Data":"088ade913c88c64edbab210f637782291157e9caed2f01086a7c4cdcf4d7c943"} Sep 19 09:51:56.189265 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.189245 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-pllt7" event={"ID":"6bd97875-f073-455a-bbaa-17fa3c598cbe","Type":"ContainerStarted","Data":"947e12d72ca2faf37de25c5dbd9d5b8ec5ba4d2a1fbaf8217182253b7307aee9"} Sep 19 09:51:56.190773 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.190750 2568 generic.go:356] "Generic (PLEG): container finished" podID="8f492656-f7cf-4308-ab65-fe7b5f59c31b" containerID="ad4671d3b63e56ba6e64ad952ed268092c4dff3f78b09bf4f8d1d862d9f5d917" exitCode=0 Sep 19 09:51:56.190868 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.190780 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bxkqq" event={"ID":"8f492656-f7cf-4308-ab65-fe7b5f59c31b","Type":"ContainerDied","Data":"ad4671d3b63e56ba6e64ad952ed268092c4dff3f78b09bf4f8d1d862d9f5d917"} Sep 19 09:51:56.192067 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.191383 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-24d99" podStartSLOduration=3.987116995 podStartE2EDuration="22.191372289s" totalImagesPullingTime="18.204255294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:51:56.191279204 +0000 UTC m=+22.598790888" watchObservedRunningTime="2026-09-19 09:51:56.191372289 +0000 UTC m=+22.598883994" Sep 19 09:51:56.192067 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.191448 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-28.ec2.internal" podStartSLOduration=21.191443103 podStartE2EDuration="21.191443103s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:51:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:51:39.154315384 +0000 UTC m=+5.561827053" watchObservedRunningTime="2026-09-19 09:51:56.191443103 +0000 UTC m=+22.598954772" Sep 19 09:51:56.207999 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.207952 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q9lm4" podStartSLOduration=4.038821706 podStartE2EDuration="22.20793574s" totalImagesPullingTime="18.169114034s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:51:56.207647128 +0000 UTC m=+22.615158799" watchObservedRunningTime="2026-09-19 09:51:56.20793574 +0000 UTC m=+22.615447409" Sep 19 09:51:56.239671 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.239617 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-pllt7" podStartSLOduration=3.921676529 podStartE2EDuration="22.239597092s" totalImagesPullingTime="18.317920563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:51:56.239328413 +0000 UTC m=+22.646840103" watchObservedRunningTime="2026-09-19 09:51:56.239597092 +0000 UTC m=+22.647108765" Sep 19 09:51:56.252634 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.252583 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-79pp9" podStartSLOduration=4.073631113 podStartE2EDuration="22.252565946s" totalImagesPullingTime="18.178934833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:51:56.252320805 +0000 UTC m=+22.659832474" watchObservedRunningTime="2026-09-19 09:51:56.252565946 +0000 UTC m=+22.660077615" Sep 19 09:51:56.266326 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.266282 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jddfm" podStartSLOduration=4.051173491 podStartE2EDuration="22.26626673s" totalImagesPullingTime="18.215093239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:51:56.266207465 +0000 UTC m=+22.673719160" watchObservedRunningTime="2026-09-19 09:51:56.26626673 +0000 UTC m=+22.673778380" Sep 19 09:51:56.317394 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:56.317370 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 19 09:51:57.050846 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:57.050729 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T09:51:56.317389488Z","UUID":"95fa2d63-8628-44f2-8edc-dc0c6192c751","Handler":null,"Name":"","Endpoint":""} Sep 19 09:51:57.052224 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:57.052204 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 19 09:51:57.052371 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:57.052239 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 19 09:51:57.080019 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:57.079990 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:57.080199 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:57.080131 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:57.196720 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:57.196684 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" event={"ID":"223993d8-8704-4f44-a51b-89729f591c1a","Type":"ContainerStarted","Data":"8ed425dbf98133b79aee7ec24f466dcc0dce580a1b4dec4ff39da86076ac5d22"} Sep 19 09:51:57.198639 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:57.198606 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rlb2k" event={"ID":"a0c22579-3d34-441c-afc8-7200ae008c73","Type":"ContainerStarted","Data":"f8e8be2dc66a8c526e4aa6cacecf620ac9bc174e172ca59ad31291b7f11c6c9e"} Sep 19 09:51:58.080404 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:58.080372 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:51:58.080559 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:58.080414 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:51:58.080559 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:58.080510 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:51:58.080746 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:58.080719 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:51:58.204033 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:58.204002 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" event={"ID":"223993d8-8704-4f44-a51b-89729f591c1a","Type":"ContainerStarted","Data":"af599d33ead85c947685dd6641ae6588620bcac8410a9883d8887b8b1c6c1cb7"} Sep 19 09:51:58.208407 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:58.208369 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" event={"ID":"ac5b2015-06bf-4a21-9879-965fc0529731","Type":"ContainerStarted","Data":"91347cc563e7c6bceaa7bfd555bb09f1d48eca78259d5a90b71330f67579c138"} Sep 19 09:51:58.221411 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:58.221319 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rlb2k" podStartSLOduration=6.018532135 podStartE2EDuration="24.221303782s" totalImagesPullingTime="18.202771647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:51:57.219427414 +0000 UTC m=+23.626939102" watchObservedRunningTime="2026-09-19 09:51:58.221303782 +0000 UTC m=+24.628815451" Sep 19 09:51:58.879027 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:58.878987 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-79pp9" Sep 19 09:51:58.879994 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:58.879967 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-79pp9" Sep 19 09:51:58.893981 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:58.893933 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2hk77" podStartSLOduration=5.137512935 podStartE2EDuration="24.893915773s" totalImagesPullingTime="19.756402838s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:51:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:51:58.221107547 +0000 UTC m=+24.628619232" watchObservedRunningTime="2026-09-19 09:51:58.893915773 +0000 UTC m=+25.301427442" Sep 19 09:51:59.080753 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:59.080718 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:51:59.080934 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:51:59.080862 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:51:59.212125 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:59.212047 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-79pp9" Sep 19 09:51:59.212553 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:51:59.212534 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-79pp9" Sep 19 09:52:00.080122 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:00.080092 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:52:00.080318 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:00.080244 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:52:00.080318 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:00.080302 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:52:00.080421 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:00.080396 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:52:01.080956 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:01.080743 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:52:01.081811 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:01.081051 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:52:01.216107 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:01.216072 2568 generic.go:356] "Generic (PLEG): container finished" podID="8f492656-f7cf-4308-ab65-fe7b5f59c31b" containerID="f002708eed264df46d50cd234416ef97fc288147e1a8ecfdc0cff4bb9e327d6a" exitCode=0 Sep 19 09:52:01.216312 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:01.216175 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bxkqq" event={"ID":"8f492656-f7cf-4308-ab65-fe7b5f59c31b","Type":"ContainerDied","Data":"f002708eed264df46d50cd234416ef97fc288147e1a8ecfdc0cff4bb9e327d6a"} Sep 19 09:52:01.220179 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:01.220152 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" event={"ID":"ac5b2015-06bf-4a21-9879-965fc0529731","Type":"ContainerStarted","Data":"440687f08609411708c105ea38fc9c1afbb98c82514bf6311e2c619902aee244"} Sep 19 09:52:01.256328 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:01.256275 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" podStartSLOduration=9.044315143 podStartE2EDuration="27.25625977s" totalImagesPullingTime="18.211944627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:01.255129957 +0000 UTC m=+27.662641625" watchObservedRunningTime="2026-09-19 09:52:01.25625977 +0000 UTC m=+27.663771438" Sep 19 09:52:02.079844 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.079816 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:52:02.079953 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.079816 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:52:02.079953 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:02.079942 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:52:02.080062 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:02.080013 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:52:02.225439 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.225236 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bxkqq" event={"ID":"8f492656-f7cf-4308-ab65-fe7b5f59c31b","Type":"ContainerStarted","Data":"c39c2eb6307048408cf1d05dc5188e504e932b82206c5d88caa8e9566b45fac1"} Sep 19 09:52:02.225785 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.225764 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:52:02.225968 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.225838 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:52:02.225968 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.225887 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:52:02.241662 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.241632 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:52:02.241806 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.241772 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:52:02.383487 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.383365 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pzsvj"] Sep 19 09:52:02.383621 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.383543 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:52:02.383699 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:02.383670 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:52:02.385885 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.385862 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hrrnt"] Sep 19 09:52:02.385979 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.385960 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:52:02.386048 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:02.386032 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:52:02.394353 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.394321 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kl7xh"] Sep 19 09:52:02.394509 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:02.394435 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:52:02.394575 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:02.394527 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:52:03.229547 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:03.229511 2568 generic.go:356] "Generic (PLEG): container finished" podID="8f492656-f7cf-4308-ab65-fe7b5f59c31b" containerID="c39c2eb6307048408cf1d05dc5188e504e932b82206c5d88caa8e9566b45fac1" exitCode=0 Sep 19 09:52:03.229907 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:03.229594 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bxkqq" event={"ID":"8f492656-f7cf-4308-ab65-fe7b5f59c31b","Type":"ContainerDied","Data":"c39c2eb6307048408cf1d05dc5188e504e932b82206c5d88caa8e9566b45fac1"} Sep 19 09:52:04.080948 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:04.080921 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:52:04.081101 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:04.081006 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:52:04.081101 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:04.081077 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:52:04.081261 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:04.081184 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:52:04.081261 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:04.081219 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:52:04.081357 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:04.081286 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:52:04.234816 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:04.234781 2568 generic.go:356] "Generic (PLEG): container finished" podID="8f492656-f7cf-4308-ab65-fe7b5f59c31b" containerID="99ca9a38f923e08d2b6038dc98d2f6af66edc60e189839c9fbb136915172a87b" exitCode=0 Sep 19 09:52:04.235219 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:04.234845 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bxkqq" event={"ID":"8f492656-f7cf-4308-ab65-fe7b5f59c31b","Type":"ContainerDied","Data":"99ca9a38f923e08d2b6038dc98d2f6af66edc60e189839c9fbb136915172a87b"} Sep 19 09:52:05.613808 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:05.613761 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:05.714625 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:05.714536 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:06.080833 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:06.080801 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:52:06.081000 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:06.080801 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:52:06.081000 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:06.080911 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kl7xh" podUID="99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e" Sep 19 09:52:06.081000 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:06.080806 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:52:06.081196 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:06.081008 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pzsvj" podUID="3875e600-bb82-4e1a-8e93-3afae30494f7" Sep 19 09:52:06.081196 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:06.081129 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hrrnt" podUID="0559c450-7285-4470-9fb8-12716eff996a" Sep 19 09:52:06.503994 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:06.503899 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:06.609455 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:06.609159 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:07.741495 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.741458 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:52:07.741987 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:07.741629 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:52:07.741987 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:07.741693 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs podName:0559c450-7285-4470-9fb8-12716eff996a nodeName:}" failed. No retries permitted until 2026-09-19 09:52:39.741678521 +0000 UTC m=+66.149190169 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs") pod "network-metrics-daemon-hrrnt" (UID: "0559c450-7285-4470-9fb8-12716eff996a") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 19 09:52:07.842758 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.842718 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:52:07.842926 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:07.842887 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 19 09:52:07.842992 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:07.842977 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret podName:3875e600-bb82-4e1a-8e93-3afae30494f7 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:39.842947613 +0000 UTC m=+66.250459268 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret") pod "global-pull-secret-syncer-pzsvj" (UID: "3875e600-bb82-4e1a-8e93-3afae30494f7") : object "kube-system"/"original-pull-secret" not registered Sep 19 09:52:07.847637 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.847613 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-28.ec2.internal" event="NodeReady" Sep 19 09:52:07.847784 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.847741 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 19 09:52:07.880356 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.880326 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65cd9c9977-lwsms"] Sep 19 09:52:07.908467 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.908436 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c9z9x"] Sep 19 09:52:07.908635 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.908576 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:07.911253 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.911226 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 19 09:52:07.911412 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.911391 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 19 09:52:07.911506 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.911419 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 19 09:52:07.911615 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.911598 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cxwhd\"" Sep 19 09:52:07.916357 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.916314 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 19 09:52:07.926254 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.926237 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2bz4b"] Sep 19 09:52:07.926389 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.926374 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:07.928792 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.928770 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 19 09:52:07.928909 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.928802 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 19 09:52:07.928909 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.928802 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-drp4f\"" Sep 19 09:52:07.943292 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.943259 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n96bf\" (UniqueName: \"kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf\") pod \"network-check-target-kl7xh\" (UID: \"99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e\") " pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:52:07.943434 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:07.943399 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 19 09:52:07.943434 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:07.943415 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 19 09:52:07.943434 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:07.943424 2568 projected.go:196] Error preparing data for projected volume kube-api-access-n96bf for pod openshift-network-diagnostics/network-check-target-kl7xh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:52:07.943575 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:07.943478 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf podName:99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e nodeName:}" failed. No retries permitted until 2026-09-19 09:52:39.943460037 +0000 UTC m=+66.350971686 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-n96bf" (UniqueName: "kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf") pod "network-check-target-kl7xh" (UID: "99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 19 09:52:07.948355 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.948333 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65cd9c9977-lwsms"] Sep 19 09:52:07.948355 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.948364 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2bz4b"] Sep 19 09:52:07.948532 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.948379 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c9z9x"] Sep 19 09:52:07.948532 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.948466 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2bz4b" Sep 19 09:52:07.951016 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.950988 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cnmxt\"" Sep 19 09:52:07.951127 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.951052 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 19 09:52:07.951220 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.950989 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 19 09:52:07.951374 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:07.951363 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 19 09:52:08.044177 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044076 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0163eb0a-704b-4150-a4de-6766f7a1043a-ca-trust-extracted\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.044177 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044114 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-bound-sa-token\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.044177 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044164 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0163eb0a-704b-4150-a4de-6766f7a1043a-installation-pull-secrets\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.044427 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044259 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/558cb6d2-cd50-4660-938f-b516d719e585-config-volume\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:08.044427 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044291 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0163eb0a-704b-4150-a4de-6766f7a1043a-trusted-ca\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.044427 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044325 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps242\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-kube-api-access-ps242\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.044427 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044347 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvkn4\" (UniqueName: \"kubernetes.io/projected/558cb6d2-cd50-4660-938f-b516d719e585-kube-api-access-xvkn4\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:08.044427 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044368 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3a8074b-9370-478d-831e-ed4b4b648f35-cert\") pod \"ingress-canary-2bz4b\" (UID: \"b3a8074b-9370-478d-831e-ed4b4b648f35\") " pod="openshift-ingress-canary/ingress-canary-2bz4b" Sep 19 09:52:08.044427 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044403 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0163eb0a-704b-4150-a4de-6766f7a1043a-image-registry-private-configuration\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.044427 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044421 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.044724 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044440 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-565gz\" (UniqueName: \"kubernetes.io/projected/b3a8074b-9370-478d-831e-ed4b4b648f35-kube-api-access-565gz\") pod \"ingress-canary-2bz4b\" (UID: \"b3a8074b-9370-478d-831e-ed4b4b648f35\") " pod="openshift-ingress-canary/ingress-canary-2bz4b" Sep 19 09:52:08.044724 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044497 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:08.044724 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044567 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-certificates\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.044724 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.044600 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/558cb6d2-cd50-4660-938f-b516d719e585-tmp-dir\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:08.080396 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.080346 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:52:08.080566 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.080413 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:52:08.080566 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.080418 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:52:08.083356 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.083332 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 19 09:52:08.083502 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.083339 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 19 09:52:08.083569 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.083551 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 19 09:52:08.083569 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.083559 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bcv4q\"" Sep 19 09:52:08.083668 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.083614 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mh4r4\"" Sep 19 09:52:08.083668 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.083644 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 19 09:52:08.144995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.144956 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0163eb0a-704b-4150-a4de-6766f7a1043a-installation-pull-secrets\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.145199 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145014 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/558cb6d2-cd50-4660-938f-b516d719e585-config-volume\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:08.145199 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145159 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0163eb0a-704b-4150-a4de-6766f7a1043a-trusted-ca\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.145326 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145205 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps242\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-kube-api-access-ps242\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.145326 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145236 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvkn4\" (UniqueName: \"kubernetes.io/projected/558cb6d2-cd50-4660-938f-b516d719e585-kube-api-access-xvkn4\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:08.145326 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145258 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3a8074b-9370-478d-831e-ed4b4b648f35-cert\") pod \"ingress-canary-2bz4b\" (UID: \"b3a8074b-9370-478d-831e-ed4b4b648f35\") " pod="openshift-ingress-canary/ingress-canary-2bz4b" Sep 19 09:52:08.145326 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145284 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0163eb0a-704b-4150-a4de-6766f7a1043a-image-registry-private-configuration\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.145326 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145309 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.145528 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145330 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-565gz\" (UniqueName: \"kubernetes.io/projected/b3a8074b-9370-478d-831e-ed4b4b648f35-kube-api-access-565gz\") pod \"ingress-canary-2bz4b\" (UID: \"b3a8074b-9370-478d-831e-ed4b4b648f35\") " pod="openshift-ingress-canary/ingress-canary-2bz4b" Sep 19 09:52:08.145528 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145356 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:08.145528 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145390 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-certificates\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.145528 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145418 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/558cb6d2-cd50-4660-938f-b516d719e585-tmp-dir\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:08.145528 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145506 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0163eb0a-704b-4150-a4de-6766f7a1043a-ca-trust-extracted\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.145783 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145529 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-bound-sa-token\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.145783 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.145560 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/558cb6d2-cd50-4660-938f-b516d719e585-config-volume\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:08.145783 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:08.145746 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 19 09:52:08.145783 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:08.145762 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65cd9c9977-lwsms: secret "image-registry-tls" not found Sep 19 09:52:08.145954 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:08.145820 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls podName:0163eb0a-704b-4150-a4de-6766f7a1043a nodeName:}" failed. No retries permitted until 2026-09-19 09:52:08.645799987 +0000 UTC m=+35.053311649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls") pod "image-registry-65cd9c9977-lwsms" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a") : secret "image-registry-tls" not found Sep 19 09:52:08.146334 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.146303 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-certificates\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.146696 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:08.146564 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 09:52:08.146696 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:08.146625 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls podName:558cb6d2-cd50-4660-938f-b516d719e585 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:08.646607156 +0000 UTC m=+35.054118817 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls") pod "dns-default-c9z9x" (UID: "558cb6d2-cd50-4660-938f-b516d719e585") : secret "dns-default-metrics-tls" not found Sep 19 09:52:08.146907 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.146882 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/558cb6d2-cd50-4660-938f-b516d719e585-tmp-dir\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:08.146907 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.146897 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0163eb0a-704b-4150-a4de-6766f7a1043a-ca-trust-extracted\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.147040 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.146940 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0163eb0a-704b-4150-a4de-6766f7a1043a-trusted-ca\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.150335 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.150317 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b3a8074b-9370-478d-831e-ed4b4b648f35-cert\") pod \"ingress-canary-2bz4b\" (UID: \"b3a8074b-9370-478d-831e-ed4b4b648f35\") " pod="openshift-ingress-canary/ingress-canary-2bz4b" Sep 19 09:52:08.150418 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.150332 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0163eb0a-704b-4150-a4de-6766f7a1043a-image-registry-private-configuration\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.150418 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.150345 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0163eb0a-704b-4150-a4de-6766f7a1043a-installation-pull-secrets\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.154666 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.154646 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvkn4\" (UniqueName: \"kubernetes.io/projected/558cb6d2-cd50-4660-938f-b516d719e585-kube-api-access-xvkn4\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:08.155009 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.154989 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps242\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-kube-api-access-ps242\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.155070 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.155011 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-565gz\" (UniqueName: \"kubernetes.io/projected/b3a8074b-9370-478d-831e-ed4b4b648f35-kube-api-access-565gz\") pod \"ingress-canary-2bz4b\" (UID: \"b3a8074b-9370-478d-831e-ed4b4b648f35\") " pod="openshift-ingress-canary/ingress-canary-2bz4b" Sep 19 09:52:08.155395 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.155376 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-bound-sa-token\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.258405 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.258367 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2bz4b" Sep 19 09:52:08.649089 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.649060 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:08.649216 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.649101 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:08.649261 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:08.649215 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 19 09:52:08.649261 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:08.649231 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65cd9c9977-lwsms: secret "image-registry-tls" not found Sep 19 09:52:08.649335 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:08.649256 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 09:52:08.649335 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:08.649279 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls podName:0163eb0a-704b-4150-a4de-6766f7a1043a nodeName:}" failed. No retries permitted until 2026-09-19 09:52:09.64926532 +0000 UTC m=+36.056776971 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls") pod "image-registry-65cd9c9977-lwsms" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a") : secret "image-registry-tls" not found Sep 19 09:52:08.649335 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:08.649302 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls podName:558cb6d2-cd50-4660-938f-b516d719e585 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:09.649288522 +0000 UTC m=+36.056800195 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls") pod "dns-default-c9z9x" (UID: "558cb6d2-cd50-4660-938f-b516d719e585") : secret "dns-default-metrics-tls" not found Sep 19 09:52:08.662908 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:08.662881 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2bz4b"] Sep 19 09:52:08.666773 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:08.666751 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3a8074b_9370_478d_831e_ed4b4b648f35.slice/crio-789df1027977ad00c484bccdb72130d8b66554e6e7eca917b4057225e2f87c6e WatchSource:0}: Error finding container 789df1027977ad00c484bccdb72130d8b66554e6e7eca917b4057225e2f87c6e: Status 404 returned error can't find the container with id 789df1027977ad00c484bccdb72130d8b66554e6e7eca917b4057225e2f87c6e Sep 19 09:52:09.141660 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:09.141422 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:09.202732 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:09.202601 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:09.250111 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:09.250080 2568 generic.go:356] "Generic (PLEG): container finished" podID="8f492656-f7cf-4308-ab65-fe7b5f59c31b" containerID="3ac5da53e95c90b7bbad4c33c137723e4e383b23e12f4d9ef1b58356f13cab85" exitCode=0 Sep 19 09:52:09.250308 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:09.250172 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bxkqq" event={"ID":"8f492656-f7cf-4308-ab65-fe7b5f59c31b","Type":"ContainerDied","Data":"3ac5da53e95c90b7bbad4c33c137723e4e383b23e12f4d9ef1b58356f13cab85"} Sep 19 09:52:09.251505 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:09.251471 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2bz4b" event={"ID":"b3a8074b-9370-478d-831e-ed4b4b648f35","Type":"ContainerStarted","Data":"789df1027977ad00c484bccdb72130d8b66554e6e7eca917b4057225e2f87c6e"} Sep 19 09:52:09.657832 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:09.657791 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:09.658014 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:09.657835 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:09.658014 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:09.657969 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 19 09:52:09.658014 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:09.657992 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65cd9c9977-lwsms: secret "image-registry-tls" not found Sep 19 09:52:09.658204 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:09.658013 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 09:52:09.658204 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:09.658063 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls podName:0163eb0a-704b-4150-a4de-6766f7a1043a nodeName:}" failed. No retries permitted until 2026-09-19 09:52:11.658043902 +0000 UTC m=+38.065555551 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls") pod "image-registry-65cd9c9977-lwsms" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a") : secret "image-registry-tls" not found Sep 19 09:52:09.658204 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:09.658083 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls podName:558cb6d2-cd50-4660-938f-b516d719e585 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:11.658073429 +0000 UTC m=+38.065585076 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls") pod "dns-default-c9z9x" (UID: "558cb6d2-cd50-4660-938f-b516d719e585") : secret "dns-default-metrics-tls" not found Sep 19 09:52:10.258060 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:10.258029 2568 generic.go:356] "Generic (PLEG): container finished" podID="8f492656-f7cf-4308-ab65-fe7b5f59c31b" containerID="f0319936ef6e0ea805f493a66a5db21678594384be5c8080ef0fb384bfab6230" exitCode=0 Sep 19 09:52:10.258579 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:10.258074 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bxkqq" event={"ID":"8f492656-f7cf-4308-ab65-fe7b5f59c31b","Type":"ContainerDied","Data":"f0319936ef6e0ea805f493a66a5db21678594384be5c8080ef0fb384bfab6230"} Sep 19 09:52:11.264766 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:11.264736 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bxkqq" event={"ID":"8f492656-f7cf-4308-ab65-fe7b5f59c31b","Type":"ContainerStarted","Data":"8f847fa65027a34599fd9adbe29b8df16f8b43a22a8cb5edbdcfbe3b6deca02d"} Sep 19 09:52:11.289603 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:11.289556 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bxkqq" podStartSLOduration=8.966674355 podStartE2EDuration="37.289538874s" totalImagesPullingTime="28.322864519s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:51:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:11.287730054 +0000 UTC m=+37.695241744" watchObservedRunningTime="2026-09-19 09:52:11.289538874 +0000 UTC m=+37.697050543" Sep 19 09:52:11.412510 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:11.412474 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:11.468187 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:11.468156 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:11.677177 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:11.677123 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:11.677177 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:11.677179 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:11.677448 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:11.677279 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 19 09:52:11.677448 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:11.677289 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 09:52:11.677448 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:11.677297 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65cd9c9977-lwsms: secret "image-registry-tls" not found Sep 19 09:52:11.677448 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:11.677341 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls podName:558cb6d2-cd50-4660-938f-b516d719e585 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:15.677328967 +0000 UTC m=+42.084840614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls") pod "dns-default-c9z9x" (UID: "558cb6d2-cd50-4660-938f-b516d719e585") : secret "dns-default-metrics-tls" not found Sep 19 09:52:11.677448 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:11.677354 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls podName:0163eb0a-704b-4150-a4de-6766f7a1043a nodeName:}" failed. No retries permitted until 2026-09-19 09:52:15.677348063 +0000 UTC m=+42.084859709 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls") pod "image-registry-65cd9c9977-lwsms" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a") : secret "image-registry-tls" not found Sep 19 09:52:12.268501 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:12.268468 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2bz4b" event={"ID":"b3a8074b-9370-478d-831e-ed4b4b648f35","Type":"ContainerStarted","Data":"e56009d9ffe234ec1fc6980871e113e6b35a4d04af7ce56c25c1cb3be1a318d7"} Sep 19 09:52:12.285516 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:12.285468 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2bz4b" podStartSLOduration=2.71673484 podStartE2EDuration="5.285451373s" totalImagesPullingTime="2.568716533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:12.284559486 +0000 UTC m=+38.692071171" watchObservedRunningTime="2026-09-19 09:52:12.285451373 +0000 UTC m=+38.692963042" Sep 19 09:52:13.597015 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:13.596981 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:13.699548 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:13.699517 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:14.694202 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:14.694119 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:14.794859 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:14.794809 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:15.534726 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:15.534689 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:15.610526 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:15.610484 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:15.667184 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:15.667129 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:15.709478 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:15.709432 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:15.709918 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:15.709491 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:15.709918 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:15.709665 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 09:52:15.709918 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:15.709734 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls podName:558cb6d2-cd50-4660-938f-b516d719e585 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:23.709714607 +0000 UTC m=+50.117226259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls") pod "dns-default-c9z9x" (UID: "558cb6d2-cd50-4660-938f-b516d719e585") : secret "dns-default-metrics-tls" not found Sep 19 09:52:15.710307 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:15.710286 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 19 09:52:15.710473 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:15.710462 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65cd9c9977-lwsms: secret "image-registry-tls" not found Sep 19 09:52:15.710600 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:15.710585 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls podName:0163eb0a-704b-4150-a4de-6766f7a1043a nodeName:}" failed. No retries permitted until 2026-09-19 09:52:23.710567897 +0000 UTC m=+50.118079545 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls") pod "image-registry-65cd9c9977-lwsms" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a") : secret "image-registry-tls" not found Sep 19 09:52:15.768162 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:15.768103 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:15.821744 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:15.821696 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:16.129675 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.129581 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:16.130290 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.130272 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g8sm5"] Sep 19 09:52:16.156975 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.156945 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g8sm5"] Sep 19 09:52:16.157165 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.157077 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.159396 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.159370 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 19 09:52:16.159552 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.159375 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 19 09:52:16.159552 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.159407 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 19 09:52:16.159748 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.159709 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qhw8x\"" Sep 19 09:52:16.159823 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.159758 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 19 09:52:16.235450 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.235405 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:16.313081 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.313041 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ds6v\" (UniqueName: \"kubernetes.io/projected/c9cbd823-b438-485f-b798-60658d28fd38-kube-api-access-4ds6v\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.313270 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.313158 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.313270 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.313222 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c9cbd823-b438-485f-b798-60658d28fd38-crio-socket\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.313270 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.313258 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9cbd823-b438-485f-b798-60658d28fd38-data-volume\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.313411 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.313280 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.332492 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.332461 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:16.414654 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.414575 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c9cbd823-b438-485f-b798-60658d28fd38-crio-socket\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.414654 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.414622 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9cbd823-b438-485f-b798-60658d28fd38-data-volume\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.414654 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.414648 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.414857 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:16.414724 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 09:52:16.414857 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.414797 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c9cbd823-b438-485f-b798-60658d28fd38-crio-socket\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.414857 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:16.414815 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm podName:c9cbd823-b438-485f-b798-60658d28fd38 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:16.914787073 +0000 UTC m=+43.322298722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g8sm5" (UID: "c9cbd823-b438-485f-b798-60658d28fd38") : configmap "kube-rbac-proxy" not found Sep 19 09:52:16.414977 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.414877 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ds6v\" (UniqueName: \"kubernetes.io/projected/c9cbd823-b438-485f-b798-60658d28fd38-kube-api-access-4ds6v\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.414977 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.414937 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.415053 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.414994 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c9cbd823-b438-485f-b798-60658d28fd38-data-volume\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.415053 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:16.415037 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 09:52:16.415153 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:16.415122 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls podName:c9cbd823-b438-485f-b798-60658d28fd38 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:16.9151109 +0000 UTC m=+43.322622546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g8sm5" (UID: "c9cbd823-b438-485f-b798-60658d28fd38") : secret "insights-runtime-extractor-tls" not found Sep 19 09:52:16.426102 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.426079 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ds6v\" (UniqueName: \"kubernetes.io/projected/c9cbd823-b438-485f-b798-60658d28fd38-kube-api-access-4ds6v\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.918017 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.917982 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.918407 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:16.918087 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:16.918407 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:16.918114 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 09:52:16.918407 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:16.918197 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 09:52:16.918407 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:16.918212 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm podName:c9cbd823-b438-485f-b798-60658d28fd38 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:17.918195511 +0000 UTC m=+44.325707161 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g8sm5" (UID: "c9cbd823-b438-485f-b798-60658d28fd38") : configmap "kube-rbac-proxy" not found Sep 19 09:52:16.918407 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:16.918239 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls podName:c9cbd823-b438-485f-b798-60658d28fd38 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:17.918228985 +0000 UTC m=+44.325740631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g8sm5" (UID: "c9cbd823-b438-485f-b798-60658d28fd38") : secret "insights-runtime-extractor-tls" not found Sep 19 09:52:17.071467 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.071425 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:17.169487 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.169394 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:17.827262 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.827229 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-mhltz"] Sep 19 09:52:17.859349 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.859309 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-mhltz"] Sep 19 09:52:17.859518 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.859424 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" Sep 19 09:52:17.862084 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.862063 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Sep 19 09:52:17.862084 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.862065 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Sep 19 09:52:17.862996 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.862965 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gl2qs\"" Sep 19 09:52:17.863091 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.863067 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Sep 19 09:52:17.863246 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.863230 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Sep 19 09:52:17.925438 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.925400 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:17.925876 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:17.925529 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 09:52:17.925876 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.925527 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:17.925876 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:17.925590 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm podName:c9cbd823-b438-485f-b798-60658d28fd38 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:19.925575614 +0000 UTC m=+46.333087261 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g8sm5" (UID: "c9cbd823-b438-485f-b798-60658d28fd38") : configmap "kube-rbac-proxy" not found Sep 19 09:52:17.925876 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:17.925629 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 09:52:17.925876 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:17.925692 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls podName:c9cbd823-b438-485f-b798-60658d28fd38 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:19.925676255 +0000 UTC m=+46.333187922 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g8sm5" (UID: "c9cbd823-b438-485f-b798-60658d28fd38") : secret "insights-runtime-extractor-tls" not found Sep 19 09:52:17.933937 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:17.933909 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:18.026564 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.026525 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl7p9\" (UniqueName: \"kubernetes.io/projected/fed51613-81c7-4e1a-a269-03e078c344c8-kube-api-access-zl7p9\") pod \"service-ca-55f6fc7b86-mhltz\" (UID: \"fed51613-81c7-4e1a-a269-03e078c344c8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" Sep 19 09:52:18.026564 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.026564 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fed51613-81c7-4e1a-a269-03e078c344c8-signing-cabundle\") pod \"service-ca-55f6fc7b86-mhltz\" (UID: \"fed51613-81c7-4e1a-a269-03e078c344c8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" Sep 19 09:52:18.026811 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.026700 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fed51613-81c7-4e1a-a269-03e078c344c8-signing-key\") pod \"service-ca-55f6fc7b86-mhltz\" (UID: \"fed51613-81c7-4e1a-a269-03e078c344c8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" Sep 19 09:52:18.032598 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.032571 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:18.128160 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.128049 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fed51613-81c7-4e1a-a269-03e078c344c8-signing-key\") pod \"service-ca-55f6fc7b86-mhltz\" (UID: \"fed51613-81c7-4e1a-a269-03e078c344c8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" Sep 19 09:52:18.128307 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.128157 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zl7p9\" (UniqueName: \"kubernetes.io/projected/fed51613-81c7-4e1a-a269-03e078c344c8-kube-api-access-zl7p9\") pod \"service-ca-55f6fc7b86-mhltz\" (UID: \"fed51613-81c7-4e1a-a269-03e078c344c8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" Sep 19 09:52:18.128307 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.128195 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fed51613-81c7-4e1a-a269-03e078c344c8-signing-cabundle\") pod \"service-ca-55f6fc7b86-mhltz\" (UID: \"fed51613-81c7-4e1a-a269-03e078c344c8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" Sep 19 09:52:18.128871 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.128851 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fed51613-81c7-4e1a-a269-03e078c344c8-signing-cabundle\") pod \"service-ca-55f6fc7b86-mhltz\" (UID: \"fed51613-81c7-4e1a-a269-03e078c344c8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" Sep 19 09:52:18.130694 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.130677 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fed51613-81c7-4e1a-a269-03e078c344c8-signing-key\") pod \"service-ca-55f6fc7b86-mhltz\" (UID: \"fed51613-81c7-4e1a-a269-03e078c344c8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" Sep 19 09:52:18.135416 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.135393 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl7p9\" (UniqueName: \"kubernetes.io/projected/fed51613-81c7-4e1a-a269-03e078c344c8-kube-api-access-zl7p9\") pod \"service-ca-55f6fc7b86-mhltz\" (UID: \"fed51613-81c7-4e1a-a269-03e078c344c8\") " pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" Sep 19 09:52:18.168339 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.168302 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" Sep 19 09:52:18.449108 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.449027 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-mhltz"] Sep 19 09:52:18.452862 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:18.452825 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed51613_81c7_4e1a_a269_03e078c344c8.slice/crio-8861eb98ad0549076415e9c945c290763ca2469f679861a6c08080db4e306458 WatchSource:0}: Error finding container 8861eb98ad0549076415e9c945c290763ca2469f679861a6c08080db4e306458: Status 404 returned error can't find the container with id 8861eb98ad0549076415e9c945c290763ca2469f679861a6c08080db4e306458 Sep 19 09:52:18.677271 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.677232 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:18.892888 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:18.892861 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jddfm_b3d2ddbc-feb3-4edb-9713-5d1822cce2b8/dns-node-resolver/0.log" Sep 19 09:52:19.287307 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:19.287227 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" event={"ID":"fed51613-81c7-4e1a-a269-03e078c344c8","Type":"ContainerStarted","Data":"8861eb98ad0549076415e9c945c290763ca2469f679861a6c08080db4e306458"} Sep 19 09:52:19.692323 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:19.692291 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-24d99_75b47a48-c5ec-4502-b447-6c6dccf8b622/node-ca/0.log" Sep 19 09:52:19.717112 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:19.717080 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:19.773124 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:19.773078 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:19.939278 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:19.939244 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:19.939463 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:19.939310 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:19.939463 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:19.939418 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 09:52:19.939590 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:19.939502 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls podName:c9cbd823-b438-485f-b798-60658d28fd38 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:23.939480459 +0000 UTC m=+50.346992114 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g8sm5" (UID: "c9cbd823-b438-485f-b798-60658d28fd38") : secret "insights-runtime-extractor-tls" not found Sep 19 09:52:19.939590 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:19.939422 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 19 09:52:19.939590 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:19.939552 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm podName:c9cbd823-b438-485f-b798-60658d28fd38 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:23.939540497 +0000 UTC m=+50.347052151 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm") pod "insights-runtime-extractor-g8sm5" (UID: "c9cbd823-b438-485f-b798-60658d28fd38") : configmap "kube-rbac-proxy" not found Sep 19 09:52:20.492464 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:20.492374 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2bz4b_b3a8074b-9370-478d-831e-ed4b4b648f35/serve-healthcheck-canary/0.log" Sep 19 09:52:20.967862 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:20.967822 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:21.067715 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:21.067658 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:21.296336 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:21.296301 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" event={"ID":"fed51613-81c7-4e1a-a269-03e078c344c8","Type":"ContainerStarted","Data":"d27f3ccf4d871763a42b3c7fbd75b1e516022d3529b6e7a429c14b62c1a2d06b"} Sep 19 09:52:21.314871 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:21.314812 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-mhltz" podStartSLOduration=1.9629554439999999 podStartE2EDuration="4.314790693s" totalImagesPullingTime="2.351835249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:52:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:21.313700876 +0000 UTC m=+47.721212560" watchObservedRunningTime="2026-09-19 09:52:21.314790693 +0000 UTC m=+47.722302363" Sep 19 09:52:21.508512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:21.508410 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:21.608900 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:21.608866 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:21.824286 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:21.824246 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:21.945761 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:21.945714 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:22.230117 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:22.230039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:22.331243 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:22.331209 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:22.385524 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:22.385495 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:22.509686 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:22.509594 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:23.500099 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:23.500060 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:23.771733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:23.771633 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:23.771733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:23.771676 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:23.772212 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:23.771793 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 19 09:52:23.772212 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:23.771812 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65cd9c9977-lwsms: secret "image-registry-tls" not found Sep 19 09:52:23.772212 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:23.771843 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 19 09:52:23.772212 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:23.771868 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls podName:0163eb0a-704b-4150-a4de-6766f7a1043a nodeName:}" failed. No retries permitted until 2026-09-19 09:52:39.77185282 +0000 UTC m=+66.179364472 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls") pod "image-registry-65cd9c9977-lwsms" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a") : secret "image-registry-tls" not found Sep 19 09:52:23.772212 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:23.771906 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls podName:558cb6d2-cd50-4660-938f-b516d719e585 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:39.77189044 +0000 UTC m=+66.179402090 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls") pod "dns-default-c9z9x" (UID: "558cb6d2-cd50-4660-938f-b516d719e585") : secret "dns-default-metrics-tls" not found Sep 19 09:52:23.909387 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:23.909350 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:23.972943 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:23.972912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:23.973115 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:23.973019 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:23.973196 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:23.973119 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 19 09:52:23.973233 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:23.973197 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls podName:c9cbd823-b438-485f-b798-60658d28fd38 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:31.973181678 +0000 UTC m=+58.380693325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls") pod "insights-runtime-extractor-g8sm5" (UID: "c9cbd823-b438-485f-b798-60658d28fd38") : secret "insights-runtime-extractor-tls" not found Sep 19 09:52:23.973470 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:23.973450 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c9cbd823-b438-485f-b798-60658d28fd38-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:24.103046 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:24.103005 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:24.297653 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:24.297607 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:24.711091 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:24.711053 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:25.318372 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:25.318333 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:25.516297 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:25.516271 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:25.616441 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:25.616343 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:25.732504 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:25.732472 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:25.898644 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:25.898554 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:26.103513 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:26.103479 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:26.189504 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:26.189418 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:26.244005 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:26.243970 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:26.301500 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:26.301467 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:26.402457 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:26.402423 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:26.504236 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:26.504126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:26.770684 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:26.770595 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:27.168389 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:27.168351 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:27.222194 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:27.222158 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:27.303866 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:27.303837 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:27.703109 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:27.703078 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:27.802863 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:27.802829 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:27.982695 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:27.982618 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:28.042535 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:28.042494 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:28.099201 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:28.099168 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:28.396279 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:28.396250 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:28.497398 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:28.497357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:28.903554 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:28.903524 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:29.096196 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:29.096162 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:29.505777 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:29.505742 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:30.100767 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:30.100722 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:30.494732 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:30.494640 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:30.910726 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:30.910669 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:31.100004 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:31.099972 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:31.498252 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:31.498210 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:32.031706 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:32.031669 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:32.034129 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:32.034108 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c9cbd823-b438-485f-b798-60658d28fd38-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g8sm5\" (UID: \"c9cbd823-b438-485f-b798-60658d28fd38\") " pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:32.066838 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:32.066802 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g8sm5" Sep 19 09:52:32.295719 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:32.295628 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:32.339465 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:32.339436 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g8sm5"] Sep 19 09:52:32.343099 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:32.343072 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9cbd823_b438_485f_b798_60658d28fd38.slice/crio-60abe4d9189d88265e1794264627bde626d5ab4923d1c85ff511560afac5f01b WatchSource:0}: Error finding container 60abe4d9189d88265e1794264627bde626d5ab4923d1c85ff511560afac5f01b: Status 404 returned error can't find the container with id 60abe4d9189d88265e1794264627bde626d5ab4923d1c85ff511560afac5f01b Sep 19 09:52:32.499279 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:32.499230 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:32.700379 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:32.700346 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:32.894271 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:32.894211 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:33.327763 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:33.327733 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8sm5" event={"ID":"c9cbd823-b438-485f-b798-60658d28fd38","Type":"ContainerStarted","Data":"2e36644b131e6e602908116f81d8d8e006b07b6f9d69319ea5ccd6cab71ca98f"} Sep 19 09:52:33.327935 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:33.327787 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8sm5" event={"ID":"c9cbd823-b438-485f-b798-60658d28fd38","Type":"ContainerStarted","Data":"dec7a70c7d7e1ff9dcdb82cdce2da8085c493f5edd9e82a907ba68393abbefc2"} Sep 19 09:52:33.327935 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:33.327807 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8sm5" event={"ID":"c9cbd823-b438-485f-b798-60658d28fd38","Type":"ContainerStarted","Data":"60abe4d9189d88265e1794264627bde626d5ab4923d1c85ff511560afac5f01b"} Sep 19 09:52:34.247367 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:34.247337 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-55lq9" Sep 19 09:52:34.614436 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:34.614408 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:35.335517 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:35.335476 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g8sm5" event={"ID":"c9cbd823-b438-485f-b798-60658d28fd38","Type":"ContainerStarted","Data":"5fab67198b236db9a170f104e7391a42a6573f26e2830c761405fa5840c3430a"} Sep 19 09:52:35.354268 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:35.354219 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g8sm5" podStartSLOduration=17.098929651 podStartE2EDuration="19.354199779s" totalImagesPullingTime="2.255270128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:52:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:35.353419542 +0000 UTC m=+61.760931211" watchObservedRunningTime="2026-09-19 09:52:35.354199779 +0000 UTC m=+61.761711448" Sep 19 09:52:37.475747 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.475711 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28"] Sep 19 09:52:37.477814 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.477797 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28" Sep 19 09:52:37.485841 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.485818 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Sep 19 09:52:37.485979 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.485859 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-z4s9b\"" Sep 19 09:52:37.486163 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.486125 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 19 09:52:37.486274 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.486228 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 19 09:52:37.486274 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.486266 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 19 09:52:37.496663 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.496637 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28"] Sep 19 09:52:37.576660 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.576630 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhp2r\" (UniqueName: \"kubernetes.io/projected/ac21907a-da38-4286-9904-abe5f03399ed-kube-api-access-hhp2r\") pod \"managed-serviceaccount-addon-agent-5c89584b46-lmw28\" (UID: \"ac21907a-da38-4286-9904-abe5f03399ed\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28" Sep 19 09:52:37.576837 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.576675 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac21907a-da38-4286-9904-abe5f03399ed-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c89584b46-lmw28\" (UID: \"ac21907a-da38-4286-9904-abe5f03399ed\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28" Sep 19 09:52:37.586580 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.586544 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6594f4c75-kq2bf"] Sep 19 09:52:37.589839 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.589813 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-kq2bf" Sep 19 09:52:37.592365 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.592341 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 19 09:52:37.592994 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.592978 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 19 09:52:37.593050 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.592978 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-z55wk\"" Sep 19 09:52:37.597513 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.597476 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:37.611789 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.611764 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-kq2bf"] Sep 19 09:52:37.677924 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.677889 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhp2r\" (UniqueName: \"kubernetes.io/projected/ac21907a-da38-4286-9904-abe5f03399ed-kube-api-access-hhp2r\") pod \"managed-serviceaccount-addon-agent-5c89584b46-lmw28\" (UID: \"ac21907a-da38-4286-9904-abe5f03399ed\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28" Sep 19 09:52:37.677924 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.677933 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcwq5\" (UniqueName: \"kubernetes.io/projected/55bd529d-d87f-4f3c-9373-c11da5bacf17-kube-api-access-hcwq5\") pod \"downloads-6594f4c75-kq2bf\" (UID: \"55bd529d-d87f-4f3c-9373-c11da5bacf17\") " pod="openshift-console/downloads-6594f4c75-kq2bf" Sep 19 09:52:37.678129 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.677952 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac21907a-da38-4286-9904-abe5f03399ed-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c89584b46-lmw28\" (UID: \"ac21907a-da38-4286-9904-abe5f03399ed\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28" Sep 19 09:52:37.680577 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.680552 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ac21907a-da38-4286-9904-abe5f03399ed-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c89584b46-lmw28\" (UID: \"ac21907a-da38-4286-9904-abe5f03399ed\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28" Sep 19 09:52:37.686938 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.686915 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhp2r\" (UniqueName: \"kubernetes.io/projected/ac21907a-da38-4286-9904-abe5f03399ed-kube-api-access-hhp2r\") pod \"managed-serviceaccount-addon-agent-5c89584b46-lmw28\" (UID: \"ac21907a-da38-4286-9904-abe5f03399ed\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28" Sep 19 09:52:37.689350 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.689326 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:37.755418 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.755323 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:37.778619 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.778588 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcwq5\" (UniqueName: \"kubernetes.io/projected/55bd529d-d87f-4f3c-9373-c11da5bacf17-kube-api-access-hcwq5\") pod \"downloads-6594f4c75-kq2bf\" (UID: \"55bd529d-d87f-4f3c-9373-c11da5bacf17\") " pod="openshift-console/downloads-6594f4c75-kq2bf" Sep 19 09:52:37.789483 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.789451 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcwq5\" (UniqueName: \"kubernetes.io/projected/55bd529d-d87f-4f3c-9373-c11da5bacf17-kube-api-access-hcwq5\") pod \"downloads-6594f4c75-kq2bf\" (UID: \"55bd529d-d87f-4f3c-9373-c11da5bacf17\") " pod="openshift-console/downloads-6594f4c75-kq2bf" Sep 19 09:52:37.800651 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.800623 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28" Sep 19 09:52:37.899189 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:37.899120 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6594f4c75-kq2bf" Sep 19 09:52:38.015588 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:38.015501 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:38.072079 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:38.071956 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28"] Sep 19 09:52:38.074473 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:38.074398 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac21907a_da38_4286_9904_abe5f03399ed.slice/crio-2075843f8c558696a455408ecd2c89426fd1bb81ac3f17d903621bdebfd4bc9c WatchSource:0}: Error finding container 2075843f8c558696a455408ecd2c89426fd1bb81ac3f17d903621bdebfd4bc9c: Status 404 returned error can't find the container with id 2075843f8c558696a455408ecd2c89426fd1bb81ac3f17d903621bdebfd4bc9c Sep 19 09:52:38.075392 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:38.075364 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:38.168599 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:38.168565 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6594f4c75-kq2bf"] Sep 19 09:52:38.171531 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:38.171509 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55bd529d_d87f_4f3c_9373_c11da5bacf17.slice/crio-d3346b9ed8e70a20e092bf219973ce5b4df6d6778feec29cfebaa014dbac83a1 WatchSource:0}: Error finding container d3346b9ed8e70a20e092bf219973ce5b4df6d6778feec29cfebaa014dbac83a1: Status 404 returned error can't find the container with id d3346b9ed8e70a20e092bf219973ce5b4df6d6778feec29cfebaa014dbac83a1 Sep 19 09:52:38.344368 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:38.344331 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-kq2bf" event={"ID":"55bd529d-d87f-4f3c-9373-c11da5bacf17","Type":"ContainerStarted","Data":"d3346b9ed8e70a20e092bf219973ce5b4df6d6778feec29cfebaa014dbac83a1"} Sep 19 09:52:38.345468 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:38.345446 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28" event={"ID":"ac21907a-da38-4286-9904-abe5f03399ed","Type":"ContainerStarted","Data":"2075843f8c558696a455408ecd2c89426fd1bb81ac3f17d903621bdebfd4bc9c"} Sep 19 09:52:39.795571 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.795523 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:52:39.796043 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.795630 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:39.796043 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.795657 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:39.798952 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.798710 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 19 09:52:39.799115 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.799090 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/558cb6d2-cd50-4660-938f-b516d719e585-metrics-tls\") pod \"dns-default-c9z9x\" (UID: \"558cb6d2-cd50-4660-938f-b516d719e585\") " pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:39.799206 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.799189 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls\") pod \"image-registry-65cd9c9977-lwsms\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:39.808758 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.808698 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0559c450-7285-4470-9fb8-12716eff996a-metrics-certs\") pod \"network-metrics-daemon-hrrnt\" (UID: \"0559c450-7285-4470-9fb8-12716eff996a\") " pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:52:39.896783 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.896365 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:52:39.899176 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.899130 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 19 09:52:39.901367 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.901341 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bcv4q\"" Sep 19 09:52:39.909765 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.909734 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hrrnt" Sep 19 09:52:39.910057 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.910039 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3875e600-bb82-4e1a-8e93-3afae30494f7-original-pull-secret\") pod \"global-pull-secret-syncer-pzsvj\" (UID: \"3875e600-bb82-4e1a-8e93-3afae30494f7\") " pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:52:39.996822 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.996787 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n96bf\" (UniqueName: \"kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf\") pod \"network-check-target-kl7xh\" (UID: \"99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e\") " pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:52:39.999436 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:39.999409 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 19 09:52:40.009810 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:40.009775 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 19 09:52:40.021113 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:40.021081 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n96bf\" (UniqueName: \"kubernetes.io/projected/99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e-kube-api-access-n96bf\") pod \"network-check-target-kl7xh\" (UID: \"99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e\") " pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:52:40.023705 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:40.023678 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-cxwhd\"" Sep 19 09:52:40.031640 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:40.031610 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:40.039441 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:40.039418 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-drp4f\"" Sep 19 09:52:40.047774 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:40.047690 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:40.196921 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:40.196889 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mh4r4\"" Sep 19 09:52:40.204760 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:40.204505 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:52:40.204760 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:40.204531 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pzsvj" Sep 19 09:52:40.219645 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:40.219617 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:40.282395 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:40.282334 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:41.038947 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.038878 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c9z9x"] Sep 19 09:52:41.042651 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:41.042615 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod558cb6d2_cd50_4660_938f_b516d719e585.slice/crio-c1f392917f7b9dfac566d52a08db9aec66fdb1311bd0161ac55bb75b5fcb5ea2 WatchSource:0}: Error finding container c1f392917f7b9dfac566d52a08db9aec66fdb1311bd0161ac55bb75b5fcb5ea2: Status 404 returned error can't find the container with id c1f392917f7b9dfac566d52a08db9aec66fdb1311bd0161ac55bb75b5fcb5ea2 Sep 19 09:52:41.053078 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.053046 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:41.059755 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.059731 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pzsvj"] Sep 19 09:52:41.063748 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:41.063699 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3875e600_bb82_4e1a_8e93_3afae30494f7.slice/crio-abd6d277e3591168316f8c7b7d33b7dccc35da457bad91a74731f5ec71f85164 WatchSource:0}: Error finding container abd6d277e3591168316f8c7b7d33b7dccc35da457bad91a74731f5ec71f85164: Status 404 returned error can't find the container with id abd6d277e3591168316f8c7b7d33b7dccc35da457bad91a74731f5ec71f85164 Sep 19 09:52:41.065586 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.065564 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kl7xh"] Sep 19 09:52:41.072568 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.071996 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hrrnt"] Sep 19 09:52:41.072752 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:41.072724 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c9fdfb_4c3d_41ae_ae9c_1a02aaddb06e.slice/crio-048f0091bc7f6ccdcd696c6aff24e83fde67e30467d6ae1b25f98cfb9640c8dc WatchSource:0}: Error finding container 048f0091bc7f6ccdcd696c6aff24e83fde67e30467d6ae1b25f98cfb9640c8dc: Status 404 returned error can't find the container with id 048f0091bc7f6ccdcd696c6aff24e83fde67e30467d6ae1b25f98cfb9640c8dc Sep 19 09:52:41.073589 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.073564 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65cd9c9977-lwsms"] Sep 19 09:52:41.077476 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:41.077440 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0559c450_7285_4470_9fb8_12716eff996a.slice/crio-a5c738f88fadafcbced4f8ca6455f78b3604004942dfe1775d52c34ea186417a WatchSource:0}: Error finding container a5c738f88fadafcbced4f8ca6455f78b3604004942dfe1775d52c34ea186417a: Status 404 returned error can't find the container with id a5c738f88fadafcbced4f8ca6455f78b3604004942dfe1775d52c34ea186417a Sep 19 09:52:41.078194 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:41.078117 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0163eb0a_704b_4150_a4de_6766f7a1043a.slice/crio-56b5edbf02d77e932bc6f7e40d4370af4b3dc384431a7198e3a210af74e1ea63 WatchSource:0}: Error finding container 56b5edbf02d77e932bc6f7e40d4370af4b3dc384431a7198e3a210af74e1ea63: Status 404 returned error can't find the container with id 56b5edbf02d77e932bc6f7e40d4370af4b3dc384431a7198e3a210af74e1ea63 Sep 19 09:52:41.117845 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.117821 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:41.218298 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.218261 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:41.357885 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.357845 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pzsvj" event={"ID":"3875e600-bb82-4e1a-8e93-3afae30494f7","Type":"ContainerStarted","Data":"abd6d277e3591168316f8c7b7d33b7dccc35da457bad91a74731f5ec71f85164"} Sep 19 09:52:41.359357 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.359323 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c9z9x" event={"ID":"558cb6d2-cd50-4660-938f-b516d719e585","Type":"ContainerStarted","Data":"c1f392917f7b9dfac566d52a08db9aec66fdb1311bd0161ac55bb75b5fcb5ea2"} Sep 19 09:52:41.360902 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.360832 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kl7xh" event={"ID":"99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e","Type":"ContainerStarted","Data":"048f0091bc7f6ccdcd696c6aff24e83fde67e30467d6ae1b25f98cfb9640c8dc"} Sep 19 09:52:41.368525 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.363844 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" event={"ID":"0163eb0a-704b-4150-a4de-6766f7a1043a","Type":"ContainerStarted","Data":"3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0"} Sep 19 09:52:41.368525 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.363921 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" event={"ID":"0163eb0a-704b-4150-a4de-6766f7a1043a","Type":"ContainerStarted","Data":"56b5edbf02d77e932bc6f7e40d4370af4b3dc384431a7198e3a210af74e1ea63"} Sep 19 09:52:41.368525 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.363980 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:52:41.368525 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.367027 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28" event={"ID":"ac21907a-da38-4286-9904-abe5f03399ed","Type":"ContainerStarted","Data":"52f1c0f4fa6a024be1222a82c481a4269bfaeff7064dbb6729e280e2dac69e54"} Sep 19 09:52:41.369925 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.369894 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hrrnt" event={"ID":"0559c450-7285-4470-9fb8-12716eff996a","Type":"ContainerStarted","Data":"a5c738f88fadafcbced4f8ca6455f78b3604004942dfe1775d52c34ea186417a"} Sep 19 09:52:41.385747 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.385689 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" podStartSLOduration=51.38566765 podStartE2EDuration="51.38566765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:51:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:41.385092659 +0000 UTC m=+67.792604353" watchObservedRunningTime="2026-09-19 09:52:41.38566765 +0000 UTC m=+67.793179318" Sep 19 09:52:41.402650 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:41.402581 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c89584b46-lmw28" podStartSLOduration=1.714259255 podStartE2EDuration="4.402559701s" totalImagesPullingTime="2.688300446s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:52:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:41.402362095 +0000 UTC m=+67.809873765" watchObservedRunningTime="2026-09-19 09:52:41.402559701 +0000 UTC m=+67.810071372" Sep 19 09:52:42.541053 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.540903 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:42.624348 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.624275 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:42.723214 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.723092 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:42.740710 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.740551 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5"] Sep 19 09:52:42.756304 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.756165 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5"] Sep 19 09:52:42.756304 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.756311 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5" Sep 19 09:52:42.762453 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.761742 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7qcsl\"" Sep 19 09:52:42.762453 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.762006 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 19 09:52:42.781859 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.781827 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:42.822314 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.822199 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9b05182f-9d1d-40e1-8c00-206a0c647071-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-wqqg5\" (UID: \"9b05182f-9d1d-40e1-8c00-206a0c647071\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5" Sep 19 09:52:42.852824 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.842006 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:42.906170 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.906121 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:42.923070 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:42.923032 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9b05182f-9d1d-40e1-8c00-206a0c647071-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-wqqg5\" (UID: \"9b05182f-9d1d-40e1-8c00-206a0c647071\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5" Sep 19 09:52:42.923268 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:42.923257 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Sep 19 09:52:42.923342 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:52:42.923329 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b05182f-9d1d-40e1-8c00-206a0c647071-tls-certificates podName:9b05182f-9d1d-40e1-8c00-206a0c647071 nodeName:}" failed. No retries permitted until 2026-09-19 09:52:43.423306228 +0000 UTC m=+69.830817875 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/9b05182f-9d1d-40e1-8c00-206a0c647071-tls-certificates") pod "prometheus-operator-admission-webhook-5c6747dc7b-wqqg5" (UID: "9b05182f-9d1d-40e1-8c00-206a0c647071") : secret "prometheus-operator-admission-webhook-tls" not found Sep 19 09:52:43.099524 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:43.099375 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:43.201959 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:43.201881 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:43.426541 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:43.426454 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9b05182f-9d1d-40e1-8c00-206a0c647071-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-wqqg5\" (UID: \"9b05182f-9d1d-40e1-8c00-206a0c647071\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5" Sep 19 09:52:43.430383 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:43.430322 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9b05182f-9d1d-40e1-8c00-206a0c647071-tls-certificates\") pod \"prometheus-operator-admission-webhook-5c6747dc7b-wqqg5\" (UID: \"9b05182f-9d1d-40e1-8c00-206a0c647071\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5" Sep 19 09:52:43.673111 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:43.672333 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5" Sep 19 09:52:44.006698 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:44.006618 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5"] Sep 19 09:52:44.012212 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:44.012179 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b05182f_9d1d_40e1_8c00_206a0c647071.slice/crio-a06e8ba23a47e36b0d03810ebdc69d4f29ce78afd785f202dc1a1a35574abb97 WatchSource:0}: Error finding container a06e8ba23a47e36b0d03810ebdc69d4f29ce78afd785f202dc1a1a35574abb97: Status 404 returned error can't find the container with id a06e8ba23a47e36b0d03810ebdc69d4f29ce78afd785f202dc1a1a35574abb97 Sep 19 09:52:44.127234 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:44.127197 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:44.186588 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:44.186550 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:44.386246 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:44.386205 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hrrnt" event={"ID":"0559c450-7285-4470-9fb8-12716eff996a","Type":"ContainerStarted","Data":"fa299687a6a8a637c96d0ee2a4e43aeb78c49d803d694a9029759cc03250bc3e"} Sep 19 09:52:44.387908 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:44.387874 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5" event={"ID":"9b05182f-9d1d-40e1-8c00-206a0c647071","Type":"ContainerStarted","Data":"a06e8ba23a47e36b0d03810ebdc69d4f29ce78afd785f202dc1a1a35574abb97"} Sep 19 09:52:45.776395 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.776354 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56d95c698c-bw2lp"] Sep 19 09:52:45.778940 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.778915 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.781555 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.781536 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b6mb7\"" Sep 19 09:52:45.781555 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.781546 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 19 09:52:45.782229 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.782210 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 19 09:52:45.782614 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.782595 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 19 09:52:45.782797 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.782780 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 19 09:52:45.782965 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.782944 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 19 09:52:45.790732 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.790694 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56d95c698c-bw2lp"] Sep 19 09:52:45.845169 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.845114 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clrvs\" (UniqueName: \"kubernetes.io/projected/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-kube-api-access-clrvs\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.845352 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.845178 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-oauth-config\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.845352 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.845257 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-serving-cert\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.845352 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.845284 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-config\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.845352 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.845310 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-oauth-serving-cert\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.845541 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.845379 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-service-ca\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.946582 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.946534 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-serving-cert\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.946753 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.946587 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-config\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.946753 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.946618 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-oauth-serving-cert\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.946753 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.946654 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-service-ca\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.946753 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.946705 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clrvs\" (UniqueName: \"kubernetes.io/projected/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-kube-api-access-clrvs\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.946753 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.946726 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-oauth-config\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.947814 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.947776 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-oauth-serving-cert\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.947814 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.947776 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-service-ca\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.948317 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.948292 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-config\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.949760 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.949735 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-oauth-config\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.950349 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.950327 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-serving-cert\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:45.955839 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:45.955816 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clrvs\" (UniqueName: \"kubernetes.io/projected/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-kube-api-access-clrvs\") pod \"console-56d95c698c-bw2lp\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:46.091604 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:46.091565 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:52:46.112100 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:46.112061 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:46.166927 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:46.166853 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:46.985604 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:46.985571 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:47.039739 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:47.039695 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:47.571338 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:47.571306 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56d95c698c-bw2lp"] Sep 19 09:52:47.575608 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:47.575571 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc43aa6f0_8bfb_4db9_af39_7efe94ae686a.slice/crio-5d45e6d523e6e4e6cc8a1616ddbe48fe45bf1dbd001981aed53805c5422d7c81 WatchSource:0}: Error finding container 5d45e6d523e6e4e6cc8a1616ddbe48fe45bf1dbd001981aed53805c5422d7c81: Status 404 returned error can't find the container with id 5d45e6d523e6e4e6cc8a1616ddbe48fe45bf1dbd001981aed53805c5422d7c81 Sep 19 09:52:48.407489 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.407445 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pzsvj" event={"ID":"3875e600-bb82-4e1a-8e93-3afae30494f7","Type":"ContainerStarted","Data":"0238a6c4e6f6563451b4f1d2cb90ebe7065fef14ddaa947eed28b5235b1b57a8"} Sep 19 09:52:48.411251 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.410591 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c9z9x" event={"ID":"558cb6d2-cd50-4660-938f-b516d719e585","Type":"ContainerStarted","Data":"db103eab8f36c78b1a52843fcfa5eb499b9551a203780e4f48b1819fc47a1b42"} Sep 19 09:52:48.411251 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.410642 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c9z9x" event={"ID":"558cb6d2-cd50-4660-938f-b516d719e585","Type":"ContainerStarted","Data":"47250ee33d142b202e7c8757927d84c7a352381362529802f8ec1b8f92d12ba8"} Sep 19 09:52:48.411251 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.411169 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c9z9x" Sep 19 09:52:48.415773 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.415024 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5" event={"ID":"9b05182f-9d1d-40e1-8c00-206a0c647071","Type":"ContainerStarted","Data":"945e75b86949dc7d676edee96f45531636c650b32dac681063f6e6f74519a91a"} Sep 19 09:52:48.415773 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.415712 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5" Sep 19 09:52:48.418319 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.418263 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d95c698c-bw2lp" event={"ID":"c43aa6f0-8bfb-4db9-af39-7efe94ae686a","Type":"ContainerStarted","Data":"5d45e6d523e6e4e6cc8a1616ddbe48fe45bf1dbd001981aed53805c5422d7c81"} Sep 19 09:52:48.420238 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.420210 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kl7xh" event={"ID":"99c9fdfb-4c3d-41ae-ae9c-1a02aaddb06e","Type":"ContainerStarted","Data":"dcf74d5ea81a9adf40c12f547fb9115a12431e2fc2a5af56979125d9608cf11b"} Sep 19 09:52:48.420457 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.420438 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:52:48.423003 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.422962 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5" Sep 19 09:52:48.425666 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.425482 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hrrnt" event={"ID":"0559c450-7285-4470-9fb8-12716eff996a","Type":"ContainerStarted","Data":"1e2534fc6b2f9a2d31a3a063a8725279b1ae43eb143a83e484d09b4ccbb41c88"} Sep 19 09:52:48.427986 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.427658 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pzsvj" podStartSLOduration=68.204529287 podStartE2EDuration="1m14.427639845s" totalImagesPullingTime="6.223110558s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 09:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:48.426251521 +0000 UTC m=+74.833763191" watchObservedRunningTime="2026-09-19 09:52:48.427639845 +0000 UTC m=+74.835151516" Sep 19 09:52:48.455220 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.454684 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5c6747dc7b-wqqg5" podStartSLOduration=3.131474409 podStartE2EDuration="6.454643575s" totalImagesPullingTime="3.323169166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:52:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:48.452753457 +0000 UTC m=+74.860265130" watchObservedRunningTime="2026-09-19 09:52:48.454643575 +0000 UTC m=+74.862155245" Sep 19 09:52:48.489698 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.489641 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hrrnt" podStartSLOduration=71.94182053 podStartE2EDuration="1m14.489617816s" totalImagesPullingTime="2.547797286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:48.470062335 +0000 UTC m=+74.877574212" watchObservedRunningTime="2026-09-19 09:52:48.489617816 +0000 UTC m=+74.897129486" Sep 19 09:52:48.490670 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.490605 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kl7xh" podStartSLOduration=68.275729919 podStartE2EDuration="1m14.490592165s" totalImagesPullingTime="6.214862246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:51:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:48.487760857 +0000 UTC m=+74.895272565" watchObservedRunningTime="2026-09-19 09:52:48.490592165 +0000 UTC m=+74.898103836" Sep 19 09:52:48.505818 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.505765 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-c9z9x" podStartSLOduration=38.52884666 podStartE2EDuration="41.505746802s" totalImagesPullingTime="2.976900142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:52:48.50489083 +0000 UTC m=+74.912402520" watchObservedRunningTime="2026-09-19 09:52:48.505746802 +0000 UTC m=+74.913258472" Sep 19 09:52:48.596411 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.596358 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:48.670017 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.669897 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:48.757493 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.757448 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:48.824730 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.823484 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p"] Sep 19 09:52:48.826497 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.825993 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:48.830913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.830067 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 19 09:52:48.830913 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.830094 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 19 09:52:48.832287 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.831891 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 19 09:52:48.832287 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.832094 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 19 09:52:48.832957 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.832607 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jck26\"" Sep 19 09:52:48.832957 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.832809 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 19 09:52:48.835403 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.835321 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:48.837577 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.837550 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p"] Sep 19 09:52:48.877499 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.877459 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/83d661af-a331-4da5-9998-6afba67a012f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-tgn9p\" (UID: \"83d661af-a331-4da5-9998-6afba67a012f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:48.877687 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.877528 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83d661af-a331-4da5-9998-6afba67a012f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-tgn9p\" (UID: \"83d661af-a331-4da5-9998-6afba67a012f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:48.877687 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.877558 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d661af-a331-4da5-9998-6afba67a012f-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-tgn9p\" (UID: \"83d661af-a331-4da5-9998-6afba67a012f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:48.877687 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.877607 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw598\" (UniqueName: \"kubernetes.io/projected/83d661af-a331-4da5-9998-6afba67a012f-kube-api-access-nw598\") pod \"prometheus-operator-7bd5d56d6-tgn9p\" (UID: \"83d661af-a331-4da5-9998-6afba67a012f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:48.890717 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.890558 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:48.979291 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.978383 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw598\" (UniqueName: \"kubernetes.io/projected/83d661af-a331-4da5-9998-6afba67a012f-kube-api-access-nw598\") pod \"prometheus-operator-7bd5d56d6-tgn9p\" (UID: \"83d661af-a331-4da5-9998-6afba67a012f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:48.979291 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.978475 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/83d661af-a331-4da5-9998-6afba67a012f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-tgn9p\" (UID: \"83d661af-a331-4da5-9998-6afba67a012f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:48.979291 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.978521 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83d661af-a331-4da5-9998-6afba67a012f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-tgn9p\" (UID: \"83d661af-a331-4da5-9998-6afba67a012f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:48.979291 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.978552 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d661af-a331-4da5-9998-6afba67a012f-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-tgn9p\" (UID: \"83d661af-a331-4da5-9998-6afba67a012f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:48.979614 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.979460 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d661af-a331-4da5-9998-6afba67a012f-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-tgn9p\" (UID: \"83d661af-a331-4da5-9998-6afba67a012f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:48.986429 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.986376 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83d661af-a331-4da5-9998-6afba67a012f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-tgn9p\" (UID: \"83d661af-a331-4da5-9998-6afba67a012f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:48.987080 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.987041 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/83d661af-a331-4da5-9998-6afba67a012f-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-tgn9p\" (UID: \"83d661af-a331-4da5-9998-6afba67a012f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:48.997759 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:48.997710 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw598\" (UniqueName: \"kubernetes.io/projected/83d661af-a331-4da5-9998-6afba67a012f-kube-api-access-nw598\") pod \"prometheus-operator-7bd5d56d6-tgn9p\" (UID: \"83d661af-a331-4da5-9998-6afba67a012f\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:49.141705 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:49.141668 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" Sep 19 09:52:49.186162 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:49.186052 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:49.243017 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:49.242882 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:49.447330 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:49.446744 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p"] Sep 19 09:52:50.844979 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:52:50.844943 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83d661af_a331_4da5_9998_6afba67a012f.slice/crio-ffe92ee4b733df5d4425bc66cbdda6245e52992bb4bf5c17801efcfd021b7057 WatchSource:0}: Error finding container ffe92ee4b733df5d4425bc66cbdda6245e52992bb4bf5c17801efcfd021b7057: Status 404 returned error can't find the container with id ffe92ee4b733df5d4425bc66cbdda6245e52992bb4bf5c17801efcfd021b7057 Sep 19 09:52:51.444940 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:51.444901 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" event={"ID":"83d661af-a331-4da5-9998-6afba67a012f","Type":"ContainerStarted","Data":"ffe92ee4b733df5d4425bc66cbdda6245e52992bb4bf5c17801efcfd021b7057"} Sep 19 09:52:52.408118 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:52.408080 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:52.461287 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:52.461199 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:52.744485 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:52.744344 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:52.797743 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:52.797700 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:53.028950 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:53.028856 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:53.096860 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:53.096823 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:52:59.442206 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:52:59.442170 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c9z9x" Sep 19 09:53:00.483596 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:00.483371 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6594f4c75-kq2bf" event={"ID":"55bd529d-d87f-4f3c-9373-c11da5bacf17","Type":"ContainerStarted","Data":"cb92a828e7d810a8da343c0c2623c158da75c5fd5b1d3558759a6602b5dc05c4"} Sep 19 09:53:00.484054 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:00.484004 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6594f4c75-kq2bf" Sep 19 09:53:00.485717 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:00.485675 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d95c698c-bw2lp" event={"ID":"c43aa6f0-8bfb-4db9-af39-7efe94ae686a","Type":"ContainerStarted","Data":"f68441a09b86801c2b49203be8f5547d8380521296ca5ea7369f0a7204967fe8"} Sep 19 09:53:00.496064 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:00.496037 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6594f4c75-kq2bf" Sep 19 09:53:00.502205 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:00.502155 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6594f4c75-kq2bf" podStartSLOduration=2.2291056830000002 podStartE2EDuration="23.502124674s" totalImagesPullingTime="21.273018991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:52:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:00.501178355 +0000 UTC m=+86.908690043" watchObservedRunningTime="2026-09-19 09:53:00.502124674 +0000 UTC m=+86.909636343" Sep 19 09:53:00.518921 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:00.517604 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56d95c698c-bw2lp" podStartSLOduration=3.364747737 podStartE2EDuration="15.517565649s" totalImagesPullingTime="12.152817912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:52:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:00.517325966 +0000 UTC m=+86.924837687" watchObservedRunningTime="2026-09-19 09:53:00.517565649 +0000 UTC m=+86.925077323" Sep 19 09:53:00.831595 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:00.831556 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:01.493303 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:01.493262 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" event={"ID":"83d661af-a331-4da5-9998-6afba67a012f","Type":"ContainerStarted","Data":"faf71919b565fb3e9cb6221859c1e6a20b4581b6f27d36d6b8a24773d492660a"} Sep 19 09:53:01.493803 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:01.493366 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" event={"ID":"83d661af-a331-4da5-9998-6afba67a012f","Type":"ContainerStarted","Data":"3c2734aa5b31cbd1302141fa50fe054800582d0df1902fe1ee903bd74b477a3d"} Sep 19 09:53:01.748954 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:01.748851 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-tgn9p" podStartSLOduration=4.08374706 podStartE2EDuration="13.748791573s" totalImagesPullingTime="9.665044513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:52:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:01.514861268 +0000 UTC m=+87.922372940" watchObservedRunningTime="2026-09-19 09:53:01.748791573 +0000 UTC m=+88.156303245" Sep 19 09:53:01.749749 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:01.749723 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56d95c698c-bw2lp"] Sep 19 09:53:01.840300 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:01.840254 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:01.941569 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:01.941530 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:01.997038 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:01.996993 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:02.111016 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:02.110984 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:02.165114 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:02.165074 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:02.380479 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:02.380405 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:53:02.531105 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:02.531068 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:02.583711 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:02.583669 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:02.904440 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:02.904362 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:03.123991 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.123950 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:03.153605 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.153564 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5"] Sep 19 09:53:03.219152 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.219039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:03.240890 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.240856 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.241157 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.241118 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-z556b"] Sep 19 09:53:03.245757 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.245269 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hd8vv\"" Sep 19 09:53:03.245757 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.245496 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Sep 19 09:53:03.245757 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.245272 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Sep 19 09:53:03.259201 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.258291 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-nv4ww"] Sep 19 09:53:03.266641 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.263612 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.272060 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.269481 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 19 09:53:03.272060 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.271976 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-l22ld\"" Sep 19 09:53:03.274149 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.274110 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 19 09:53:03.281045 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.275810 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 19 09:53:03.293265 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.293232 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:03.302085 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.302054 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f72d1c86-a71e-4109-af38-e690580b09e3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.302299 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.302099 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f72d1c86-a71e-4109-af38-e690580b09e3-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.302299 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.302131 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9fmg\" (UniqueName: \"kubernetes.io/projected/f72d1c86-a71e-4109-af38-e690580b09e3-kube-api-access-k9fmg\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.302299 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.302196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f72d1c86-a71e-4109-af38-e690580b09e3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.314015 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.313979 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5"] Sep 19 09:53:03.314265 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.314252 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-z556b"] Sep 19 09:53:03.314485 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.314474 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.318161 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.317447 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 19 09:53:03.318161 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.317720 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 19 09:53:03.318161 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.317891 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 19 09:53:03.318161 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.318048 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xqtq7\"" Sep 19 09:53:03.403116 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.403084 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.403358 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.403341 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f72d1c86-a71e-4109-af38-e690580b09e3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.403449 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.403439 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-root\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.403525 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.403513 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmwsk\" (UniqueName: \"kubernetes.io/projected/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-kube-api-access-fmwsk\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.403585 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.403577 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-sys\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.403651 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.403643 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.403714 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.403705 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-accelerators-collector-config\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.403774 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.403765 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5tw2\" (UniqueName: \"kubernetes.io/projected/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-kube-api-access-n5tw2\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.403832 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.403824 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.403947 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.403937 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.404166 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.404073 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f72d1c86-a71e-4109-af38-e690580b09e3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.404166 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.404113 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.404166 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.404165 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f72d1c86-a71e-4109-af38-e690580b09e3-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.404362 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.404190 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-textfile\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.404362 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.404213 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-tls\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.404362 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.404237 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.404362 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.404269 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9fmg\" (UniqueName: \"kubernetes.io/projected/f72d1c86-a71e-4109-af38-e690580b09e3-kube-api-access-k9fmg\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.404362 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.404296 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-wtmp\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.404362 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.404327 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-metrics-client-ca\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.404774 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:53:03.404755 2568 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Sep 19 09:53:03.404921 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:53:03.404909 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f72d1c86-a71e-4109-af38-e690580b09e3-openshift-state-metrics-tls podName:f72d1c86-a71e-4109-af38-e690580b09e3 nodeName:}" failed. No retries permitted until 2026-09-19 09:53:03.904888709 +0000 UTC m=+90.312400370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f72d1c86-a71e-4109-af38-e690580b09e3-openshift-state-metrics-tls") pod "openshift-state-metrics-86c5bc79d6-v7dq5" (UID: "f72d1c86-a71e-4109-af38-e690580b09e3") : secret "openshift-state-metrics-tls" not found Sep 19 09:53:03.405010 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.404969 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f72d1c86-a71e-4109-af38-e690580b09e3-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.409878 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.409846 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f72d1c86-a71e-4109-af38-e690580b09e3-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.413843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.413819 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9fmg\" (UniqueName: \"kubernetes.io/projected/f72d1c86-a71e-4109-af38-e690580b09e3-kube-api-access-k9fmg\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.453722 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.453668 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.504725 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-metrics-client-ca\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.504774 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.504815 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-root\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.504844 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmwsk\" (UniqueName: \"kubernetes.io/projected/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-kube-api-access-fmwsk\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.504868 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-sys\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.504893 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.504918 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-accelerators-collector-config\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.504942 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5tw2\" (UniqueName: \"kubernetes.io/projected/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-kube-api-access-n5tw2\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.504967 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.505004 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.505059 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.505083 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-textfile\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.505105 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-tls\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.505127 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.505179 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-wtmp\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.505969 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.505343 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-wtmp\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.506844 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.505907 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-metrics-client-ca\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.506844 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.506488 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-root\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.506844 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.506703 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-sys\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.507035 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.507010 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-textfile\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.507331 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:53:03.507308 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Sep 19 09:53:03.507433 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:53:03.507377 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-tls podName:f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe nodeName:}" failed. No retries permitted until 2026-09-19 09:53:04.007357574 +0000 UTC m=+90.414869224 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-tls") pod "node-exporter-nv4ww" (UID: "f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe") : secret "node-exporter-tls" not found Sep 19 09:53:03.508242 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.508215 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-accelerators-collector-config\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.509257 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.509230 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.509758 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.509731 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.510589 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.510567 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.510845 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.510826 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.511438 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.511414 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.514834 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.514724 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5tw2\" (UniqueName: \"kubernetes.io/projected/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-kube-api-access-n5tw2\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.514834 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.514809 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmwsk\" (UniqueName: \"kubernetes.io/projected/35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c-kube-api-access-fmwsk\") pod \"kube-state-metrics-84c885658c-z556b\" (UID: \"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.526556 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.526524 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:03.555092 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.554976 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:03.594117 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.594080 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" Sep 19 09:53:03.671206 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.671084 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:03.705860 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.705827 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65cd9c9977-lwsms"] Sep 19 09:53:03.787500 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.787417 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:03.873494 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.873452 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:03.908116 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.908075 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f72d1c86-a71e-4109-af38-e690580b09e3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.911126 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.911096 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f72d1c86-a71e-4109-af38-e690580b09e3-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-v7dq5\" (UID: \"f72d1c86-a71e-4109-af38-e690580b09e3\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:03.949394 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.949352 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-z556b"] Sep 19 09:53:03.952534 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:03.952507 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:03.954700 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:53:03.954672 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35ac82b9_a6b6_4e8c_8c95_eaf19d23ef0c.slice/crio-16c44dd2568afa6f3aaf10e1eac1d1eafb094e3ff96f6d393d99ab23562c9cd9 WatchSource:0}: Error finding container 16c44dd2568afa6f3aaf10e1eac1d1eafb094e3ff96f6d393d99ab23562c9cd9: Status 404 returned error can't find the container with id 16c44dd2568afa6f3aaf10e1eac1d1eafb094e3ff96f6d393d99ab23562c9cd9 Sep 19 09:53:04.009258 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.009221 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-tls\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:04.012253 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.012222 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe-node-exporter-tls\") pod \"node-exporter-nv4ww\" (UID: \"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe\") " pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:04.130010 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.129803 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:04.158048 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.158009 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" Sep 19 09:53:04.225837 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.225797 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:04.228000 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.227970 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-nv4ww" Sep 19 09:53:04.372417 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.372366 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:04.486721 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.486636 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5"] Sep 19 09:53:04.491516 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:53:04.491483 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf72d1c86_a71e_4109_af38_e690580b09e3.slice/crio-a605461eaa0e95a60b51de1fd70141d9b36dfa0c2d872976828616dc0d42fec5 WatchSource:0}: Error finding container a605461eaa0e95a60b51de1fd70141d9b36dfa0c2d872976828616dc0d42fec5: Status 404 returned error can't find the container with id a605461eaa0e95a60b51de1fd70141d9b36dfa0c2d872976828616dc0d42fec5 Sep 19 09:53:04.510244 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.510209 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nv4ww" event={"ID":"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe","Type":"ContainerStarted","Data":"5600af25ea26909c02518324e223754c9eef3c218e2a0aae72b517fe1cda848c"} Sep 19 09:53:04.511782 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.511748 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" event={"ID":"f72d1c86-a71e-4109-af38-e690580b09e3","Type":"ContainerStarted","Data":"a605461eaa0e95a60b51de1fd70141d9b36dfa0c2d872976828616dc0d42fec5"} Sep 19 09:53:04.513357 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.513196 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" event={"ID":"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c","Type":"ContainerStarted","Data":"16c44dd2568afa6f3aaf10e1eac1d1eafb094e3ff96f6d393d99ab23562c9cd9"} Sep 19 09:53:04.587683 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.587071 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:04.689042 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.688969 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:04.791022 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:04.790935 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:05.134520 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.134480 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-9ffdd76d9-dmbck"] Sep 19 09:53:05.155807 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.155775 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-9ffdd76d9-dmbck"] Sep 19 09:53:05.155964 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.155922 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.159148 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.158619 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 19 09:53:05.159148 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.158656 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-81ekn1gtcs193\"" Sep 19 09:53:05.159148 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.158706 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 19 09:53:05.159148 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.158887 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 19 09:53:05.159148 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.158916 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-mwrbx\"" Sep 19 09:53:05.159148 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.159124 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 19 09:53:05.160365 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.160339 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 19 09:53:05.220575 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.220534 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4kbr\" (UniqueName: \"kubernetes.io/projected/5e56f2cd-f694-4c15-ae09-4ba427845ad9-kube-api-access-m4kbr\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.220745 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.220589 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-grpc-tls\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.220745 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.220693 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.220745 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.220739 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-tls\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.221005 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.220908 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.221120 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.221018 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.221120 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.221070 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e56f2cd-f694-4c15-ae09-4ba427845ad9-metrics-client-ca\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.221120 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.221099 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.321894 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.321850 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4kbr\" (UniqueName: \"kubernetes.io/projected/5e56f2cd-f694-4c15-ae09-4ba427845ad9-kube-api-access-m4kbr\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.322121 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.321905 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-grpc-tls\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.322121 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.321936 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.322121 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.321958 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-tls\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.322121 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.321992 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.322121 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.322023 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.322121 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.322046 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e56f2cd-f694-4c15-ae09-4ba427845ad9-metrics-client-ca\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.322121 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.322077 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.323741 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.323231 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5e56f2cd-f694-4c15-ae09-4ba427845ad9-metrics-client-ca\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.325991 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.325652 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.325991 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.325764 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.326494 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.326472 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-tls\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.326555 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.326510 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.329234 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.329208 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.331489 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.331464 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4kbr\" (UniqueName: \"kubernetes.io/projected/5e56f2cd-f694-4c15-ae09-4ba427845ad9-kube-api-access-m4kbr\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.332182 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.332123 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5e56f2cd-f694-4c15-ae09-4ba427845ad9-secret-grpc-tls\") pod \"thanos-querier-9ffdd76d9-dmbck\" (UID: \"5e56f2cd-f694-4c15-ae09-4ba427845ad9\") " pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.467904 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.467756 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:05.520492 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.520454 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" event={"ID":"f72d1c86-a71e-4109-af38-e690580b09e3","Type":"ContainerStarted","Data":"073a63c219e98d45c7d4cef120e48a86b8035a158c716954872975b2adf7f8c0"} Sep 19 09:53:05.520688 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.520526 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" event={"ID":"f72d1c86-a71e-4109-af38-e690580b09e3","Type":"ContainerStarted","Data":"fcb7754b94d10aedb4d8a5ccdb9a84491b4e0a7f912b8d36d79584b3f19c89f4"} Sep 19 09:53:05.568448 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.568413 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:05.670261 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.670229 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:05.785625 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.785367 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:05.885053 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:05.885016 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:06.093801 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:06.093725 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:53:06.094371 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:06.094295 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:06.149995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:06.149965 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:06.247993 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:06.247943 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:06.415571 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:06.415448 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:06.512522 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:06.512481 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:06.630760 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:06.630717 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:06.895046 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:06.895006 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-9ffdd76d9-dmbck"] Sep 19 09:53:06.938884 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:53:06.936778 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e56f2cd_f694_4c15_ae09_4ba427845ad9.slice/crio-46f4c43c8eed3d6e53152559b1cbc80ae5466981c840f9162a749f2368b3553d WatchSource:0}: Error finding container 46f4c43c8eed3d6e53152559b1cbc80ae5466981c840f9162a749f2368b3553d: Status 404 returned error can't find the container with id 46f4c43c8eed3d6e53152559b1cbc80ae5466981c840f9162a749f2368b3553d Sep 19 09:53:07.531528 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.531484 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" event={"ID":"5e56f2cd-f694-4c15-ae09-4ba427845ad9","Type":"ContainerStarted","Data":"46f4c43c8eed3d6e53152559b1cbc80ae5466981c840f9162a749f2368b3553d"} Sep 19 09:53:07.533672 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.533638 2568 generic.go:356] "Generic (PLEG): container finished" podID="f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe" containerID="a86f9a0f04ab1c3b92ab41fd8cbc81f643369b87a0745ca0d5dac2ca7cd39291" exitCode=0 Sep 19 09:53:07.533927 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.533902 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nv4ww" event={"ID":"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe","Type":"ContainerDied","Data":"a86f9a0f04ab1c3b92ab41fd8cbc81f643369b87a0745ca0d5dac2ca7cd39291"} Sep 19 09:53:07.536643 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.536614 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" event={"ID":"f72d1c86-a71e-4109-af38-e690580b09e3","Type":"ContainerStarted","Data":"ac49725ab3c3309d2a8badb4aaf763142a17f60de1410cfbd53c55c81f262326"} Sep 19 09:53:07.540361 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.540331 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" event={"ID":"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c","Type":"ContainerStarted","Data":"e79303a9100780aadbb43dc4419ed06746e0fe8ebf989301202da84b1a8af428"} Sep 19 09:53:07.540458 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.540393 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" event={"ID":"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c","Type":"ContainerStarted","Data":"23c65aef0e95559d17bb7d55e7c34bd2b093d35b5b00c096d165a64fa27889d2"} Sep 19 09:53:07.540458 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.540422 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" event={"ID":"35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c","Type":"ContainerStarted","Data":"9b673bb732257acfc5d18bb15db03d663810beae4fb2a01af9dfca9f73687837"} Sep 19 09:53:07.632497 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.632435 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-v7dq5" podStartSLOduration=2.451990988 podStartE2EDuration="4.632410856s" totalImagesPullingTime="2.180419868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:07.630248725 +0000 UTC m=+94.037760430" watchObservedRunningTime="2026-09-19 09:53:07.632410856 +0000 UTC m=+94.039922527" Sep 19 09:53:07.633213 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.632806 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-z556b" podStartSLOduration=1.986595913 podStartE2EDuration="4.632795061s" totalImagesPullingTime="2.646199148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:07.585238973 +0000 UTC m=+93.992750644" watchObservedRunningTime="2026-09-19 09:53:07.632795061 +0000 UTC m=+94.040306731" Sep 19 09:53:07.961158 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.955780 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:07.965166 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.964893 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d8c4998bc-7whbw"] Sep 19 09:53:07.985176 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.984898 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d8c4998bc-7whbw"] Sep 19 09:53:07.985176 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.985028 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:07.999848 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:07.993826 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 19 09:53:08.050438 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.050275 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:08.146827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.146653 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-console-config\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.146827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.146700 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c392d429-323a-409f-b8d8-43812351c241-console-serving-cert\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.146827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.146753 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-oauth-serving-cert\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.146827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.146780 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q9vv\" (UniqueName: \"kubernetes.io/projected/c392d429-323a-409f-b8d8-43812351c241-kube-api-access-7q9vv\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.146827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.146818 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-service-ca\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.147280 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.146851 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-trusted-ca-bundle\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.147280 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.146937 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c392d429-323a-409f-b8d8-43812351c241-console-oauth-config\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.150597 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.150448 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:08.247455 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.247399 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-service-ca\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.247455 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.247462 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-trusted-ca-bundle\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.247742 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.247509 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c392d429-323a-409f-b8d8-43812351c241-console-oauth-config\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.247742 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.247560 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-console-config\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.247742 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.247587 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c392d429-323a-409f-b8d8-43812351c241-console-serving-cert\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.247742 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.247636 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-oauth-serving-cert\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.247742 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.247665 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q9vv\" (UniqueName: \"kubernetes.io/projected/c392d429-323a-409f-b8d8-43812351c241-kube-api-access-7q9vv\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.248737 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.248709 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-console-config\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.249611 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.249585 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-service-ca\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.250323 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.250294 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-oauth-serving-cert\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.251327 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.251290 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-trusted-ca-bundle\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.251681 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.251656 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c392d429-323a-409f-b8d8-43812351c241-console-serving-cert\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.251756 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.251699 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c392d429-323a-409f-b8d8-43812351c241-console-oauth-config\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.255945 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.254825 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:08.257170 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.257116 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q9vv\" (UniqueName: \"kubernetes.io/projected/c392d429-323a-409f-b8d8-43812351c241-kube-api-access-7q9vv\") pod \"console-6d8c4998bc-7whbw\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.303964 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.303862 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:08.372545 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.372513 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8567b5c8df-g9fl7"] Sep 19 09:53:08.391962 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.391599 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.392830 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.392756 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8567b5c8df-g9fl7"] Sep 19 09:53:08.400980 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.399083 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 19 09:53:08.400980 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.399376 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 19 09:53:08.400980 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.399592 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 19 09:53:08.400980 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.399827 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 19 09:53:08.400980 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.399951 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-x4jsj\"" Sep 19 09:53:08.400980 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.400046 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 19 09:53:08.406809 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.406784 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 19 09:53:08.549116 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.548999 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nv4ww" event={"ID":"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe","Type":"ContainerStarted","Data":"3ec53181435e6c7c4032fdc7b99bbfa21890013f8f1f81e79b1f21bfcb19d3c6"} Sep 19 09:53:08.549116 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.549072 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-nv4ww" event={"ID":"f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe","Type":"ContainerStarted","Data":"8807abacfd8ada09d9eb3c781b0fefa09ab8590ba514521b6764538070fae30a"} Sep 19 09:53:08.549597 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.549572 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.549693 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.549625 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-serving-certs-ca-bundle\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.549693 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.549677 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-telemeter-client-tls\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.549803 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.549706 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.549803 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.549735 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-federate-client-tls\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.549803 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.549762 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7mkd\" (UniqueName: \"kubernetes.io/projected/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-kube-api-access-j7mkd\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.549962 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.549821 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-metrics-client-ca\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.549962 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.549879 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-secret-telemeter-client\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.571916 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.571861 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-nv4ww" podStartSLOduration=3.136239371 podStartE2EDuration="5.571811768s" totalImagesPullingTime="2.325176058s" totalInitContainerRuntime="110.396339ms" isStatefulPod=true podCreationTimestamp="2026-09-19 09:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:08.570524647 +0000 UTC m=+94.978036338" watchObservedRunningTime="2026-09-19 09:53:08.571811768 +0000 UTC m=+94.979323440" Sep 19 09:53:08.631333 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.631288 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d8c4998bc-7whbw"] Sep 19 09:53:08.653916 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.651331 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.653916 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.651476 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-serving-certs-ca-bundle\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.653916 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.651539 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-telemeter-client-tls\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.653916 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.651565 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.653916 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.651600 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-federate-client-tls\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.653916 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.651642 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7mkd\" (UniqueName: \"kubernetes.io/projected/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-kube-api-access-j7mkd\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.653916 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.651713 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-metrics-client-ca\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.653916 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.651791 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-secret-telemeter-client\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.653916 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.653072 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.653916 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.653728 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-serving-certs-ca-bundle\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.655780 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.655751 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-metrics-client-ca\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.656476 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.656448 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-secret-telemeter-client\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.657713 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.657674 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-telemeter-client-tls\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.658324 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.658299 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.659389 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.659335 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-federate-client-tls\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.666614 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.666592 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7mkd\" (UniqueName: \"kubernetes.io/projected/925d61a7-0c16-4b05-9e85-6cb8c0c0518f-kube-api-access-j7mkd\") pod \"telemeter-client-8567b5c8df-g9fl7\" (UID: \"925d61a7-0c16-4b05-9e85-6cb8c0c0518f\") " pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:08.717430 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:08.717399 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" Sep 19 09:53:09.239230 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:53:09.239195 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc392d429_323a_409f_b8d8_43812351c241.slice/crio-6f2c3672f47fe5ee77e76275da374fc1b74a57129574c0e045c1e66d4a68fbe6 WatchSource:0}: Error finding container 6f2c3672f47fe5ee77e76275da374fc1b74a57129574c0e045c1e66d4a68fbe6: Status 404 returned error can't find the container with id 6f2c3672f47fe5ee77e76275da374fc1b74a57129574c0e045c1e66d4a68fbe6 Sep 19 09:53:09.381793 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.381706 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:09.496789 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.496703 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:09.528664 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.528632 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 09:53:09.547804 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.547777 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.554551 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.554526 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 19 09:53:09.554871 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.554848 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 19 09:53:09.556997 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.556978 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8567b5c8df-g9fl7"] Sep 19 09:53:09.557853 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.557815 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 19 09:53:09.557853 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.557847 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 19 09:53:09.558100 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.558084 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 19 09:53:09.558214 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.558195 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 19 09:53:09.559053 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.559029 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8c4998bc-7whbw" event={"ID":"c392d429-323a-409f-b8d8-43812351c241","Type":"ContainerStarted","Data":"00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468"} Sep 19 09:53:09.559129 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.559083 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8c4998bc-7whbw" event={"ID":"c392d429-323a-409f-b8d8-43812351c241","Type":"ContainerStarted","Data":"6f2c3672f47fe5ee77e76275da374fc1b74a57129574c0e045c1e66d4a68fbe6"} Sep 19 09:53:09.561569 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.561555 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 19 09:53:09.562690 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.562676 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6f30fcubjq98f\"" Sep 19 09:53:09.563232 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.563219 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-42gf9\"" Sep 19 09:53:09.563904 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.563885 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 19 09:53:09.564003 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.563937 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 19 09:53:09.564981 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.564966 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 19 09:53:09.565070 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.565015 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 19 09:53:09.565070 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.565034 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 19 09:53:09.565978 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.565964 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 19 09:53:09.566329 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.566313 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 19 09:53:09.573228 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.573202 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 09:53:09.574302 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.574071 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 19 09:53:09.631258 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:53:09.631230 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod925d61a7_0c16_4b05_9e85_6cb8c0c0518f.slice/crio-9ccac07ab7523838f4827b99db0bc5861dbdb37e986ebc603b03278cfae6632e WatchSource:0}: Error finding container 9ccac07ab7523838f4827b99db0bc5861dbdb37e986ebc603b03278cfae6632e: Status 404 returned error can't find the container with id 9ccac07ab7523838f4827b99db0bc5861dbdb37e986ebc603b03278cfae6632e Sep 19 09:53:09.661037 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661016 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpnw4\" (UniqueName: \"kubernetes.io/projected/38160819-b5e0-4401-bc3c-363dcafda28d-kube-api-access-kpnw4\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661162 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661047 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38160819-b5e0-4401-bc3c-363dcafda28d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661162 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661113 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661287 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661181 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-web-config\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661287 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661278 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661389 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661308 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661441 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661385 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661441 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661410 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661547 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661478 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661547 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661506 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661547 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661532 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661686 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661560 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661686 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661614 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661686 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661668 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661794 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661706 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38160819-b5e0-4401-bc3c-363dcafda28d-config-out\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661794 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661755 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661890 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661820 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-config\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661944 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661894 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.661944 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661920 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.662038 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.661947 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763366 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763282 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763366 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763324 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763366 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763341 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763366 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763366 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763667 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763393 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763667 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763415 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763667 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763440 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763667 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763462 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763667 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763667 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763521 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38160819-b5e0-4401-bc3c-363dcafda28d-config-out\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763667 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763552 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763667 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763587 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-config\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.763667 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763639 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.764095 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763673 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.764095 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763700 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.764095 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763723 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpnw4\" (UniqueName: \"kubernetes.io/projected/38160819-b5e0-4401-bc3c-363dcafda28d-kube-api-access-kpnw4\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.764095 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763756 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38160819-b5e0-4401-bc3c-363dcafda28d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.764095 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763795 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.764095 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.763822 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-web-config\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.764443 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.764205 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.764443 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.764227 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.765811 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.765483 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.765811 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.765761 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.767205 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.766575 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.767205 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.766711 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.767505 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.767482 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.767797 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.767777 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.768601 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.768582 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.768902 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.768875 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.769273 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.769254 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.771920 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.771896 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.772020 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.771860 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.772020 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.771838 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38160819-b5e0-4401-bc3c-363dcafda28d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.772414 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.772350 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38160819-b5e0-4401-bc3c-363dcafda28d-config-out\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.772673 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.772639 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-web-config\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.772673 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.772999 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.773429 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.773405 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-config\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.773757 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.773727 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.774048 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.774024 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.777117 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.777093 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpnw4\" (UniqueName: \"kubernetes.io/projected/38160819-b5e0-4401-bc3c-363dcafda28d-kube-api-access-kpnw4\") pod \"prometheus-k8s-0\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:09.861159 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:09.859121 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:10.184244 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:10.184200 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d8c4998bc-7whbw" podStartSLOduration=3.184182629 podStartE2EDuration="3.184182629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:53:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:09.65891605 +0000 UTC m=+96.066427740" watchObservedRunningTime="2026-09-19 09:53:10.184182629 +0000 UTC m=+96.591694298" Sep 19 09:53:10.184625 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:10.184612 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 09:53:10.186414 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:53:10.186393 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38160819_b5e0_4401_bc3c_363dcafda28d.slice/crio-f2e4f2c6c594727c9a1acd9c1df6691d2b111baed6eee0947b09b01e728e7b91 WatchSource:0}: Error finding container f2e4f2c6c594727c9a1acd9c1df6691d2b111baed6eee0947b09b01e728e7b91: Status 404 returned error can't find the container with id f2e4f2c6c594727c9a1acd9c1df6691d2b111baed6eee0947b09b01e728e7b91 Sep 19 09:53:10.566577 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:10.566530 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerStarted","Data":"f2e4f2c6c594727c9a1acd9c1df6691d2b111baed6eee0947b09b01e728e7b91"} Sep 19 09:53:10.570066 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:10.569902 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" event={"ID":"5e56f2cd-f694-4c15-ae09-4ba427845ad9","Type":"ContainerStarted","Data":"614d7baf0d6afb747cfb200351b0225bf144eba082cdb8fd7156b27da2ac22bf"} Sep 19 09:53:10.570066 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:10.569985 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" event={"ID":"5e56f2cd-f694-4c15-ae09-4ba427845ad9","Type":"ContainerStarted","Data":"5ad3904b351de684515d82bcdbf915bac7ed93d72985bd4de7306dcfe1dd3081"} Sep 19 09:53:10.570066 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:10.570024 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" event={"ID":"5e56f2cd-f694-4c15-ae09-4ba427845ad9","Type":"ContainerStarted","Data":"1640b349fc4146db0b2bda899c3dfe04d6a0a4f3a6bc7ba67616837716be1dbe"} Sep 19 09:53:10.571405 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:10.571379 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" event={"ID":"925d61a7-0c16-4b05-9e85-6cb8c0c0518f","Type":"ContainerStarted","Data":"9ccac07ab7523838f4827b99db0bc5861dbdb37e986ebc603b03278cfae6632e"} Sep 19 09:53:11.231497 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:11.231447 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:11.327868 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:11.327829 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:12.583004 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:12.582977 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" event={"ID":"5e56f2cd-f694-4c15-ae09-4ba427845ad9","Type":"ContainerStarted","Data":"f4dbbeaf5ef25c59c525ffa988d4e2dd43759be657b7852802d392e5d14fe568"} Sep 19 09:53:12.583360 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:12.583089 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" event={"ID":"5e56f2cd-f694-4c15-ae09-4ba427845ad9","Type":"ContainerStarted","Data":"ad458e1319158652664881449870d965191c5627e269604741d7b5668fdb0710"} Sep 19 09:53:12.584676 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:12.584653 2568 generic.go:356] "Generic (PLEG): container finished" podID="38160819-b5e0-4401-bc3c-363dcafda28d" containerID="f9e648dfe294e14d545242638857fe9910f389a5bef04b656583beb2c3d7fdc7" exitCode=0 Sep 19 09:53:12.584760 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:12.584706 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerDied","Data":"f9e648dfe294e14d545242638857fe9910f389a5bef04b656583beb2c3d7fdc7"} Sep 19 09:53:13.593448 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:13.593406 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" event={"ID":"5e56f2cd-f694-4c15-ae09-4ba427845ad9","Type":"ContainerStarted","Data":"91cc94a70bd92c8f31d761436a0fde84245bb774dc44dbe2ef90c50780a64140"} Sep 19 09:53:13.594066 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:13.593597 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:13.595608 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:13.595587 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" event={"ID":"925d61a7-0c16-4b05-9e85-6cb8c0c0518f","Type":"ContainerStarted","Data":"a73fe225713bd8265c1b245a83a7cdc4def64ed0d50175a40b8790968cbc2f86"} Sep 19 09:53:13.595699 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:13.595629 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" event={"ID":"925d61a7-0c16-4b05-9e85-6cb8c0c0518f","Type":"ContainerStarted","Data":"ce4f9c3e23eec961c900ce0d8e8e64979f2eea901629afdc865616437571eb33"} Sep 19 09:53:13.595699 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:13.595656 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" event={"ID":"925d61a7-0c16-4b05-9e85-6cb8c0c0518f","Type":"ContainerStarted","Data":"2ecdb44442c6ae5e87c55bc3323f492ff5b3fd3088bd9c5ab3ced2f11fb137c1"} Sep 19 09:53:13.621090 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:13.621035 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" podStartSLOduration=3.77227976 podStartE2EDuration="8.621018237s" totalImagesPullingTime="4.848738477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:53:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:13.619339306 +0000 UTC m=+100.026850998" watchObservedRunningTime="2026-09-19 09:53:13.621018237 +0000 UTC m=+100.028529903" Sep 19 09:53:13.651424 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:13.650282 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8567b5c8df-g9fl7" podStartSLOduration=2.7469735379999998 podStartE2EDuration="5.65025787s" totalImagesPullingTime="2.903284332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:53:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:13.649378785 +0000 UTC m=+100.056909447" watchObservedRunningTime="2026-09-19 09:53:13.65025787 +0000 UTC m=+100.057769539" Sep 19 09:53:13.760120 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:13.760081 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d8c4998bc-7whbw"] Sep 19 09:53:14.609376 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:14.609344 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-9ffdd76d9-dmbck" Sep 19 09:53:14.659662 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:14.659626 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:14.712410 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:14.712372 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:16.617546 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:16.617515 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerStarted","Data":"27ec1b5f0207e150be4449392bb907b7bc4e45633dd22c86b5bd310596263054"} Sep 19 09:53:16.617988 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:16.617581 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerStarted","Data":"e92a614dd0585e514bebca76474857c3d108b1fa61112a483a169d394f08cc50"} Sep 19 09:53:17.625382 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:17.625340 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerStarted","Data":"72718a46828351151f974499a71e542769c657f9d63d1abf3f624ebbc078edb2"} Sep 19 09:53:17.625847 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:17.625437 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerStarted","Data":"4787a322cc9a704e93ad7de696f077f4f255890098b14814f522cfb2f0530f8e"} Sep 19 09:53:17.625847 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:17.625464 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerStarted","Data":"213493e5789441f089664071a90bc1722b34f46692a9beae82ef49376cc442a2"} Sep 19 09:53:17.625847 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:17.625487 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerStarted","Data":"d2e64bc538db0503221bd2fcb3696c0cff9448f2256c6a535f441dd2ceaf6741"} Sep 19 09:53:17.662706 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:17.662599 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.931275013 podStartE2EDuration="8.662572604s" totalImagesPullingTime="5.715099403s" totalInitContainerRuntime="16.198188ms" isStatefulPod=false podCreationTimestamp="2026-09-19 09:53:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:53:17.658499271 +0000 UTC m=+104.066010966" watchObservedRunningTime="2026-09-19 09:53:17.662572604 +0000 UTC m=+104.070084274" Sep 19 09:53:18.304500 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:18.304466 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:19.432939 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:19.432908 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kl7xh" Sep 19 09:53:19.860016 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:19.859976 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:53:21.501736 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:21.501699 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:21.602649 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:21.602612 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:24.969306 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:24.969274 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:25.069624 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:25.069590 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:25.170642 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:25.170605 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:27.520735 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.520696 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56d95c698c-bw2lp" podUID="c43aa6f0-8bfb-4db9-af39-7efe94ae686a" containerName="console" containerID="cri-o://f68441a09b86801c2b49203be8f5547d8380521296ca5ea7369f0a7204967fe8" gracePeriod=15 Sep 19 09:53:27.669990 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.669962 2568 generic.go:356] "Generic (PLEG): container finished" podID="c43aa6f0-8bfb-4db9-af39-7efe94ae686a" containerID="f68441a09b86801c2b49203be8f5547d8380521296ca5ea7369f0a7204967fe8" exitCode=0 Sep 19 09:53:27.670167 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.670039 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d95c698c-bw2lp" event={"ID":"c43aa6f0-8bfb-4db9-af39-7efe94ae686a","Type":"ContainerDied","Data":"f68441a09b86801c2b49203be8f5547d8380521296ca5ea7369f0a7204967fe8"} Sep 19 09:53:27.804417 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.804390 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:53:27.936477 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.936441 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-serving-cert\") pod \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " Sep 19 09:53:27.936654 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.936498 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-oauth-serving-cert\") pod \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " Sep 19 09:53:27.936654 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.936530 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clrvs\" (UniqueName: \"kubernetes.io/projected/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-kube-api-access-clrvs\") pod \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " Sep 19 09:53:27.936654 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.936561 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-service-ca\") pod \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " Sep 19 09:53:27.936654 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.936590 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-oauth-config\") pod \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " Sep 19 09:53:27.936654 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.936630 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-config\") pod \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\" (UID: \"c43aa6f0-8bfb-4db9-af39-7efe94ae686a\") " Sep 19 09:53:27.936998 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.936938 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-service-ca" (OuterVolumeSpecName: "service-ca") pod "c43aa6f0-8bfb-4db9-af39-7efe94ae686a" (UID: "c43aa6f0-8bfb-4db9-af39-7efe94ae686a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:53:27.936998 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.936951 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c43aa6f0-8bfb-4db9-af39-7efe94ae686a" (UID: "c43aa6f0-8bfb-4db9-af39-7efe94ae686a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:53:27.937181 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.937039 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-config" (OuterVolumeSpecName: "console-config") pod "c43aa6f0-8bfb-4db9-af39-7efe94ae686a" (UID: "c43aa6f0-8bfb-4db9-af39-7efe94ae686a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:53:27.939088 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.939062 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c43aa6f0-8bfb-4db9-af39-7efe94ae686a" (UID: "c43aa6f0-8bfb-4db9-af39-7efe94ae686a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:53:27.939209 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.939114 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-kube-api-access-clrvs" (OuterVolumeSpecName: "kube-api-access-clrvs") pod "c43aa6f0-8bfb-4db9-af39-7efe94ae686a" (UID: "c43aa6f0-8bfb-4db9-af39-7efe94ae686a"). InnerVolumeSpecName "kube-api-access-clrvs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:53:27.939209 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:27.939156 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c43aa6f0-8bfb-4db9-af39-7efe94ae686a" (UID: "c43aa6f0-8bfb-4db9-af39-7efe94ae686a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:53:28.037868 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:28.037786 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-oauth-config\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:28.037868 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:28.037818 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-config\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:28.037868 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:28.037827 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-console-serving-cert\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:28.037868 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:28.037835 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-oauth-serving-cert\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:28.037868 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:28.037843 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clrvs\" (UniqueName: \"kubernetes.io/projected/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-kube-api-access-clrvs\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:28.037868 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:28.037853 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c43aa6f0-8bfb-4db9-af39-7efe94ae686a-service-ca\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:28.676457 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:28.676423 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d95c698c-bw2lp" event={"ID":"c43aa6f0-8bfb-4db9-af39-7efe94ae686a","Type":"ContainerDied","Data":"5d45e6d523e6e4e6cc8a1616ddbe48fe45bf1dbd001981aed53805c5422d7c81"} Sep 19 09:53:28.676457 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:28.676449 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d95c698c-bw2lp" Sep 19 09:53:28.676974 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:28.676489 2568 scope.go:117] "RemoveContainer" containerID="f68441a09b86801c2b49203be8f5547d8380521296ca5ea7369f0a7204967fe8" Sep 19 09:53:28.694717 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:28.694683 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56d95c698c-bw2lp"] Sep 19 09:53:28.698190 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:28.698160 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56d95c698c-bw2lp"] Sep 19 09:53:28.761305 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:28.761263 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" podUID="0163eb0a-704b-4150-a4de-6766f7a1043a" containerName="registry" containerID="cri-o://3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0" gracePeriod=30 Sep 19 09:53:29.050504 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.050463 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:53:29.149181 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.149125 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0163eb0a-704b-4150-a4de-6766f7a1043a-image-registry-private-configuration\") pod \"0163eb0a-704b-4150-a4de-6766f7a1043a\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " Sep 19 09:53:29.149181 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.149184 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0163eb0a-704b-4150-a4de-6766f7a1043a-trusted-ca\") pod \"0163eb0a-704b-4150-a4de-6766f7a1043a\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " Sep 19 09:53:29.149438 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.149206 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0163eb0a-704b-4150-a4de-6766f7a1043a-ca-trust-extracted\") pod \"0163eb0a-704b-4150-a4de-6766f7a1043a\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " Sep 19 09:53:29.149438 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.149222 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-certificates\") pod \"0163eb0a-704b-4150-a4de-6766f7a1043a\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " Sep 19 09:53:29.149438 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.149244 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls\") pod \"0163eb0a-704b-4150-a4de-6766f7a1043a\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " Sep 19 09:53:29.149438 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.149261 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0163eb0a-704b-4150-a4de-6766f7a1043a-installation-pull-secrets\") pod \"0163eb0a-704b-4150-a4de-6766f7a1043a\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " Sep 19 09:53:29.149438 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.149334 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps242\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-kube-api-access-ps242\") pod \"0163eb0a-704b-4150-a4de-6766f7a1043a\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " Sep 19 09:53:29.149438 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.149371 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-bound-sa-token\") pod \"0163eb0a-704b-4150-a4de-6766f7a1043a\" (UID: \"0163eb0a-704b-4150-a4de-6766f7a1043a\") " Sep 19 09:53:29.149757 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.149726 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0163eb0a-704b-4150-a4de-6766f7a1043a" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:53:29.149938 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.149832 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0163eb0a-704b-4150-a4de-6766f7a1043a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0163eb0a-704b-4150-a4de-6766f7a1043a" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:53:29.152055 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.152028 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0163eb0a-704b-4150-a4de-6766f7a1043a" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:53:29.152209 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.152070 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0163eb0a-704b-4150-a4de-6766f7a1043a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0163eb0a-704b-4150-a4de-6766f7a1043a" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:53:29.152310 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.152282 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0163eb0a-704b-4150-a4de-6766f7a1043a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0163eb0a-704b-4150-a4de-6766f7a1043a" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:53:29.152394 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.152336 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-kube-api-access-ps242" (OuterVolumeSpecName: "kube-api-access-ps242") pod "0163eb0a-704b-4150-a4de-6766f7a1043a" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a"). InnerVolumeSpecName "kube-api-access-ps242". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:53:29.152394 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.152379 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0163eb0a-704b-4150-a4de-6766f7a1043a" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:53:29.157869 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.157841 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0163eb0a-704b-4150-a4de-6766f7a1043a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0163eb0a-704b-4150-a4de-6766f7a1043a" (UID: "0163eb0a-704b-4150-a4de-6766f7a1043a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:53:29.250580 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.250494 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0163eb0a-704b-4150-a4de-6766f7a1043a-image-registry-private-configuration\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:29.250580 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.250528 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0163eb0a-704b-4150-a4de-6766f7a1043a-trusted-ca\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:29.250580 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.250540 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0163eb0a-704b-4150-a4de-6766f7a1043a-ca-trust-extracted\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:29.250580 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.250548 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-certificates\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:29.250580 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.250557 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-registry-tls\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:29.250580 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.250565 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0163eb0a-704b-4150-a4de-6766f7a1043a-installation-pull-secrets\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:29.250580 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.250573 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps242\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-kube-api-access-ps242\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:29.250580 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.250581 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0163eb0a-704b-4150-a4de-6766f7a1043a-bound-sa-token\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:29.682061 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.682027 2568 generic.go:356] "Generic (PLEG): container finished" podID="0163eb0a-704b-4150-a4de-6766f7a1043a" containerID="3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0" exitCode=0 Sep 19 09:53:29.682648 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.682076 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" Sep 19 09:53:29.682648 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.682107 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" event={"ID":"0163eb0a-704b-4150-a4de-6766f7a1043a","Type":"ContainerDied","Data":"3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0"} Sep 19 09:53:29.682648 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.682181 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65cd9c9977-lwsms" event={"ID":"0163eb0a-704b-4150-a4de-6766f7a1043a","Type":"ContainerDied","Data":"56b5edbf02d77e932bc6f7e40d4370af4b3dc384431a7198e3a210af74e1ea63"} Sep 19 09:53:29.682648 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.682203 2568 scope.go:117] "RemoveContainer" containerID="3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0" Sep 19 09:53:29.691404 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.691386 2568 scope.go:117] "RemoveContainer" containerID="3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0" Sep 19 09:53:29.691721 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:53:29.691698 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0\": container with ID starting with 3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0 not found: ID does not exist" containerID="3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0" Sep 19 09:53:29.691786 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.691730 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0"} err="failed to get container status \"3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0\": rpc error: code = NotFound desc = could not find container \"3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0\": container with ID starting with 3f487f063e6b1898d9ff2aacd7111d2c0fdab63b0b947a303cbab39fd30b08c0 not found: ID does not exist" Sep 19 09:53:29.702789 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.702763 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65cd9c9977-lwsms"] Sep 19 09:53:29.706150 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:29.706113 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65cd9c9977-lwsms"] Sep 19 09:53:30.085963 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:30.085931 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0163eb0a-704b-4150-a4de-6766f7a1043a" path="/var/lib/kubelet/pods/0163eb0a-704b-4150-a4de-6766f7a1043a/volumes" Sep 19 09:53:30.086341 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:30.086328 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c43aa6f0-8bfb-4db9-af39-7efe94ae686a" path="/var/lib/kubelet/pods/c43aa6f0-8bfb-4db9-af39-7efe94ae686a/volumes" Sep 19 09:53:36.502689 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:36.502659 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:36.555246 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:36.555203 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:38.786191 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:38.786104 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d8c4998bc-7whbw" podUID="c392d429-323a-409f-b8d8-43812351c241" containerName="console" containerID="cri-o://00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468" gracePeriod=15 Sep 19 09:53:39.075158 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.075104 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:39.128297 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.128261 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-oauth-serving-cert\") pod \"c392d429-323a-409f-b8d8-43812351c241\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " Sep 19 09:53:39.128297 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.128293 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-trusted-ca-bundle\") pod \"c392d429-323a-409f-b8d8-43812351c241\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " Sep 19 09:53:39.128543 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.128414 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c392d429-323a-409f-b8d8-43812351c241-console-oauth-config\") pod \"c392d429-323a-409f-b8d8-43812351c241\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " Sep 19 09:53:39.128543 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.128475 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7q9vv\" (UniqueName: \"kubernetes.io/projected/c392d429-323a-409f-b8d8-43812351c241-kube-api-access-7q9vv\") pod \"c392d429-323a-409f-b8d8-43812351c241\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " Sep 19 09:53:39.128543 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.128498 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-service-ca\") pod \"c392d429-323a-409f-b8d8-43812351c241\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " Sep 19 09:53:39.128701 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.128673 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c392d429-323a-409f-b8d8-43812351c241-console-serving-cert\") pod \"c392d429-323a-409f-b8d8-43812351c241\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " Sep 19 09:53:39.128753 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.128723 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-console-config\") pod \"c392d429-323a-409f-b8d8-43812351c241\" (UID: \"c392d429-323a-409f-b8d8-43812351c241\") " Sep 19 09:53:39.128825 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.128685 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c392d429-323a-409f-b8d8-43812351c241" (UID: "c392d429-323a-409f-b8d8-43812351c241"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:53:39.128901 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.128854 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-service-ca" (OuterVolumeSpecName: "service-ca") pod "c392d429-323a-409f-b8d8-43812351c241" (UID: "c392d429-323a-409f-b8d8-43812351c241"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:53:39.129059 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.129004 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-service-ca\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:39.129059 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.129025 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-oauth-serving-cert\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:39.129199 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.129117 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-console-config" (OuterVolumeSpecName: "console-config") pod "c392d429-323a-409f-b8d8-43812351c241" (UID: "c392d429-323a-409f-b8d8-43812351c241"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:53:39.129199 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.129165 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c392d429-323a-409f-b8d8-43812351c241" (UID: "c392d429-323a-409f-b8d8-43812351c241"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:53:39.130819 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.130791 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c392d429-323a-409f-b8d8-43812351c241-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c392d429-323a-409f-b8d8-43812351c241" (UID: "c392d429-323a-409f-b8d8-43812351c241"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:53:39.130918 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.130835 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c392d429-323a-409f-b8d8-43812351c241-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c392d429-323a-409f-b8d8-43812351c241" (UID: "c392d429-323a-409f-b8d8-43812351c241"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:53:39.130918 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.130851 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c392d429-323a-409f-b8d8-43812351c241-kube-api-access-7q9vv" (OuterVolumeSpecName: "kube-api-access-7q9vv") pod "c392d429-323a-409f-b8d8-43812351c241" (UID: "c392d429-323a-409f-b8d8-43812351c241"). InnerVolumeSpecName "kube-api-access-7q9vv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:53:39.229430 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.229395 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-trusted-ca-bundle\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:39.229430 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.229426 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c392d429-323a-409f-b8d8-43812351c241-console-oauth-config\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:39.229430 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.229436 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q9vv\" (UniqueName: \"kubernetes.io/projected/c392d429-323a-409f-b8d8-43812351c241-kube-api-access-7q9vv\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:39.229672 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.229444 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c392d429-323a-409f-b8d8-43812351c241-console-serving-cert\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:39.229672 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.229452 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c392d429-323a-409f-b8d8-43812351c241-console-config\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:53:39.727606 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.727563 2568 generic.go:356] "Generic (PLEG): container finished" podID="c392d429-323a-409f-b8d8-43812351c241" containerID="00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468" exitCode=0 Sep 19 09:53:39.727843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.727614 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d8c4998bc-7whbw" Sep 19 09:53:39.727843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.727648 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8c4998bc-7whbw" event={"ID":"c392d429-323a-409f-b8d8-43812351c241","Type":"ContainerDied","Data":"00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468"} Sep 19 09:53:39.727843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.727702 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d8c4998bc-7whbw" event={"ID":"c392d429-323a-409f-b8d8-43812351c241","Type":"ContainerDied","Data":"6f2c3672f47fe5ee77e76275da374fc1b74a57129574c0e045c1e66d4a68fbe6"} Sep 19 09:53:39.727843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.727723 2568 scope.go:117] "RemoveContainer" containerID="00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468" Sep 19 09:53:39.737097 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.737075 2568 scope.go:117] "RemoveContainer" containerID="00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468" Sep 19 09:53:39.737408 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:53:39.737390 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468\": container with ID starting with 00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468 not found: ID does not exist" containerID="00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468" Sep 19 09:53:39.737455 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.737419 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468"} err="failed to get container status \"00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468\": rpc error: code = NotFound desc = could not find container \"00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468\": container with ID starting with 00f90762e05284bd2c502758a1f1de309cd09867f53a9a68240aa69644096468 not found: ID does not exist" Sep 19 09:53:39.748059 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.748028 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d8c4998bc-7whbw"] Sep 19 09:53:39.752165 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:39.752123 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d8c4998bc-7whbw"] Sep 19 09:53:40.085170 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:40.085119 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c392d429-323a-409f-b8d8-43812351c241" path="/var/lib/kubelet/pods/c392d429-323a-409f-b8d8-43812351c241/volumes" Sep 19 09:53:41.843285 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:41.843253 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2bz4b_b3a8074b-9370-478d-831e-ed4b4b648f35/serve-healthcheck-canary/0.log" Sep 19 09:53:44.430748 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:44.430711 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:44.533230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:44.533194 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:53:44.587410 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:53:44.587380 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:09.860298 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:09.860266 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:09.880923 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:09.880898 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:10.871560 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:10.871535 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:19.279733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:19.279698 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:19.381054 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:19.381020 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:26.197817 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:26.197722 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:26.299294 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:26.299259 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:26.495478 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:26.495374 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:26.595832 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:26.595804 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:27.289791 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.289748 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:27.339773 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.339728 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:27.778087 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.778050 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 09:54:27.778542 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.778506 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="prometheus" containerID="cri-o://e92a614dd0585e514bebca76474857c3d108b1fa61112a483a169d394f08cc50" gracePeriod=600 Sep 19 09:54:27.778640 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.778531 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="kube-rbac-proxy-web" containerID="cri-o://213493e5789441f089664071a90bc1722b34f46692a9beae82ef49376cc442a2" gracePeriod=600 Sep 19 09:54:27.778640 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.778554 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="kube-rbac-proxy" containerID="cri-o://4787a322cc9a704e93ad7de696f077f4f255890098b14814f522cfb2f0530f8e" gracePeriod=600 Sep 19 09:54:27.778640 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.778591 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="config-reloader" containerID="cri-o://27ec1b5f0207e150be4449392bb907b7bc4e45633dd22c86b5bd310596263054" gracePeriod=600 Sep 19 09:54:27.778796 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.778540 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://72718a46828351151f974499a71e542769c657f9d63d1abf3f624ebbc078edb2" gracePeriod=600 Sep 19 09:54:27.778796 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.778530 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="thanos-sidecar" containerID="cri-o://d2e64bc538db0503221bd2fcb3696c0cff9448f2256c6a535f441dd2ceaf6741" gracePeriod=600 Sep 19 09:54:27.926733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.926700 2568 generic.go:356] "Generic (PLEG): container finished" podID="38160819-b5e0-4401-bc3c-363dcafda28d" containerID="72718a46828351151f974499a71e542769c657f9d63d1abf3f624ebbc078edb2" exitCode=0 Sep 19 09:54:27.926733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.926725 2568 generic.go:356] "Generic (PLEG): container finished" podID="38160819-b5e0-4401-bc3c-363dcafda28d" containerID="4787a322cc9a704e93ad7de696f077f4f255890098b14814f522cfb2f0530f8e" exitCode=0 Sep 19 09:54:27.926733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.926735 2568 generic.go:356] "Generic (PLEG): container finished" podID="38160819-b5e0-4401-bc3c-363dcafda28d" containerID="213493e5789441f089664071a90bc1722b34f46692a9beae82ef49376cc442a2" exitCode=0 Sep 19 09:54:27.927027 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.926743 2568 generic.go:356] "Generic (PLEG): container finished" podID="38160819-b5e0-4401-bc3c-363dcafda28d" containerID="d2e64bc538db0503221bd2fcb3696c0cff9448f2256c6a535f441dd2ceaf6741" exitCode=0 Sep 19 09:54:27.927027 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.926751 2568 generic.go:356] "Generic (PLEG): container finished" podID="38160819-b5e0-4401-bc3c-363dcafda28d" containerID="27ec1b5f0207e150be4449392bb907b7bc4e45633dd22c86b5bd310596263054" exitCode=0 Sep 19 09:54:27.927027 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.926760 2568 generic.go:356] "Generic (PLEG): container finished" podID="38160819-b5e0-4401-bc3c-363dcafda28d" containerID="e92a614dd0585e514bebca76474857c3d108b1fa61112a483a169d394f08cc50" exitCode=0 Sep 19 09:54:27.927027 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.926774 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerDied","Data":"72718a46828351151f974499a71e542769c657f9d63d1abf3f624ebbc078edb2"} Sep 19 09:54:27.927027 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.926859 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerDied","Data":"4787a322cc9a704e93ad7de696f077f4f255890098b14814f522cfb2f0530f8e"} Sep 19 09:54:27.927027 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.926905 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerDied","Data":"213493e5789441f089664071a90bc1722b34f46692a9beae82ef49376cc442a2"} Sep 19 09:54:27.927027 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.926946 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerDied","Data":"d2e64bc538db0503221bd2fcb3696c0cff9448f2256c6a535f441dd2ceaf6741"} Sep 19 09:54:27.927027 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.926986 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerDied","Data":"27ec1b5f0207e150be4449392bb907b7bc4e45633dd22c86b5bd310596263054"} Sep 19 09:54:27.927027 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:27.927026 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerDied","Data":"e92a614dd0585e514bebca76474857c3d108b1fa61112a483a169d394f08cc50"} Sep 19 09:54:28.045528 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.045503 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:28.173087 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173050 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kpnw4\" (UniqueName: \"kubernetes.io/projected/38160819-b5e0-4401-bc3c-363dcafda28d-kube-api-access-kpnw4\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173279 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173093 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-serving-certs-ca-bundle\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173279 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173131 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-0\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173279 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173243 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-2\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173462 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173282 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-trusted-ca-bundle\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173462 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173387 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-metrics-client-ca\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173462 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173422 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-kubelet-serving-ca-bundle\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173462 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173450 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-config\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173658 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173478 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-thanos-prometheus-http-client-file\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173658 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173482 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:54:28.173658 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173514 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-grpc-tls\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173658 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173545 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-web-config\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173658 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173568 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-kube-rbac-proxy\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173658 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173585 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:54:28.173658 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173595 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.173658 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173617 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:54:28.173658 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173639 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38160819-b5e0-4401-bc3c-363dcafda28d-config-out\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.174085 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173672 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38160819-b5e0-4401-bc3c-363dcafda28d-tls-assets\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.174085 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173701 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-metrics-client-certs\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.174085 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173745 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.174085 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173777 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-1\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.174085 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173779 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:54:28.174085 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173785 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:54:28.174085 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173805 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-db\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.174085 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.173876 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-tls\") pod \"38160819-b5e0-4401-bc3c-363dcafda28d\" (UID: \"38160819-b5e0-4401-bc3c-363dcafda28d\") " Sep 19 09:54:28.174489 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.174301 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.174489 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.174311 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:54:28.174489 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.174320 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.174489 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.174353 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.174489 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.174371 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-metrics-client-ca\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.174489 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.174387 2568 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.176838 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.175314 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 19 09:54:28.176838 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.176801 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:54:28.177264 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.177119 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38160819-b5e0-4401-bc3c-363dcafda28d-kube-api-access-kpnw4" (OuterVolumeSpecName: "kube-api-access-kpnw4") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "kube-api-access-kpnw4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:54:28.177264 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.177217 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38160819-b5e0-4401-bc3c-363dcafda28d-config-out" (OuterVolumeSpecName: "config-out") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:54:28.177590 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.177562 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:54:28.177812 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.177776 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-config" (OuterVolumeSpecName: "config") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:54:28.177916 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.177840 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:54:28.178183 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.178130 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:54:28.178183 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.178174 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:54:28.178358 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.178301 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:54:28.178567 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.178550 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:54:28.178837 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.178819 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:54:28.179834 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.179818 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38160819-b5e0-4401-bc3c-363dcafda28d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:54:28.190593 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.190570 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-web-config" (OuterVolumeSpecName: "web-config") pod "38160819-b5e0-4401-bc3c-363dcafda28d" (UID: "38160819-b5e0-4401-bc3c-363dcafda28d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 19 09:54:28.275270 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275234 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275270 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275265 2568 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-config\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275270 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275275 2568 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275286 2568 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-grpc-tls\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275294 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-web-config\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275302 2568 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-kube-rbac-proxy\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275310 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275318 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/38160819-b5e0-4401-bc3c-363dcafda28d-config-out\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275326 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/38160819-b5e0-4401-bc3c-363dcafda28d-tls-assets\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275333 2568 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-metrics-client-certs\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275343 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275353 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275361 2568 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/38160819-b5e0-4401-bc3c-363dcafda28d-prometheus-k8s-db\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275370 2568 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/38160819-b5e0-4401-bc3c-363dcafda28d-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.275512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.275378 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpnw4\" (UniqueName: \"kubernetes.io/projected/38160819-b5e0-4401-bc3c-363dcafda28d-kube-api-access-kpnw4\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:54:28.934812 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.934775 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"38160819-b5e0-4401-bc3c-363dcafda28d","Type":"ContainerDied","Data":"f2e4f2c6c594727c9a1acd9c1df6691d2b111baed6eee0947b09b01e728e7b91"} Sep 19 09:54:28.935243 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.934852 2568 scope.go:117] "RemoveContainer" containerID="72718a46828351151f974499a71e542769c657f9d63d1abf3f624ebbc078edb2" Sep 19 09:54:28.935243 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.934860 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:28.942604 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.942582 2568 scope.go:117] "RemoveContainer" containerID="4787a322cc9a704e93ad7de696f077f4f255890098b14814f522cfb2f0530f8e" Sep 19 09:54:28.949860 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.949842 2568 scope.go:117] "RemoveContainer" containerID="213493e5789441f089664071a90bc1722b34f46692a9beae82ef49376cc442a2" Sep 19 09:54:28.956868 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.956851 2568 scope.go:117] "RemoveContainer" containerID="d2e64bc538db0503221bd2fcb3696c0cff9448f2256c6a535f441dd2ceaf6741" Sep 19 09:54:28.958860 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.958840 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 09:54:28.964203 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.964180 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 09:54:28.965852 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.965829 2568 scope.go:117] "RemoveContainer" containerID="27ec1b5f0207e150be4449392bb907b7bc4e45633dd22c86b5bd310596263054" Sep 19 09:54:28.972708 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.972691 2568 scope.go:117] "RemoveContainer" containerID="e92a614dd0585e514bebca76474857c3d108b1fa61112a483a169d394f08cc50" Sep 19 09:54:28.979886 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.979869 2568 scope.go:117] "RemoveContainer" containerID="f9e648dfe294e14d545242638857fe9910f389a5bef04b656583beb2c3d7fdc7" Sep 19 09:54:28.989231 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989210 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 09:54:28.989491 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989479 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="config-reloader" Sep 19 09:54:28.989534 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989493 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="config-reloader" Sep 19 09:54:28.989534 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989500 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c43aa6f0-8bfb-4db9-af39-7efe94ae686a" containerName="console" Sep 19 09:54:28.989534 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989505 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c43aa6f0-8bfb-4db9-af39-7efe94ae686a" containerName="console" Sep 19 09:54:28.989534 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989513 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="thanos-sidecar" Sep 19 09:54:28.989534 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989519 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="thanos-sidecar" Sep 19 09:54:28.989534 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989530 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="prometheus" Sep 19 09:54:28.989534 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989535 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="prometheus" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989542 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c392d429-323a-409f-b8d8-43812351c241" containerName="console" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989547 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c392d429-323a-409f-b8d8-43812351c241" containerName="console" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989553 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="kube-rbac-proxy-web" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989558 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="kube-rbac-proxy-web" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989567 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0163eb0a-704b-4150-a4de-6766f7a1043a" containerName="registry" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989572 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0163eb0a-704b-4150-a4de-6766f7a1043a" containerName="registry" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989577 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="init-config-reloader" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989582 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="init-config-reloader" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989587 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="kube-rbac-proxy" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989592 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="kube-rbac-proxy" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989597 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="kube-rbac-proxy-thanos" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989603 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="kube-rbac-proxy-thanos" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989647 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0163eb0a-704b-4150-a4de-6766f7a1043a" containerName="registry" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989655 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="kube-rbac-proxy" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989664 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="kube-rbac-proxy-thanos" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989669 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c43aa6f0-8bfb-4db9-af39-7efe94ae686a" containerName="console" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989675 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="config-reloader" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989680 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c392d429-323a-409f-b8d8-43812351c241" containerName="console" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989685 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="prometheus" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989690 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="thanos-sidecar" Sep 19 09:54:28.989733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.989696 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" containerName="kube-rbac-proxy-web" Sep 19 09:54:28.992606 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.992587 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:28.995318 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.995161 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 19 09:54:28.995318 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.995172 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 19 09:54:28.995318 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.995213 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 19 09:54:28.995318 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.995227 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 19 09:54:28.995544 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.995326 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 19 09:54:28.995544 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.995346 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6f30fcubjq98f\"" Sep 19 09:54:28.995544 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.995481 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 19 09:54:28.995695 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.995677 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 19 09:54:28.995767 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.995720 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 19 09:54:28.995767 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.995742 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 19 09:54:28.995882 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.995858 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-42gf9\"" Sep 19 09:54:28.996083 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.995978 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 19 09:54:28.996083 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.996011 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 19 09:54:28.996273 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.996181 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 19 09:54:28.996341 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.996307 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 19 09:54:28.999845 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:28.999650 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 19 09:54:29.002192 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.002173 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 19 09:54:29.012649 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.012624 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 09:54:29.080817 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.080789 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.080987 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.080825 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.080987 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.080846 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.080987 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.080862 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.080987 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.080938 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8905a6a4-dc65-429d-a392-7c4f396f8acd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.080987 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.080972 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8905a6a4-dc65-429d-a392-7c4f396f8acd-config-out\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081223 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.080995 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-config\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081223 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081029 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081223 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081069 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-web-config\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081223 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081103 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081223 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081125 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081223 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081176 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081223 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081199 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081223 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081213 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081498 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081228 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081498 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081250 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081498 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081265 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081498 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081322 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrspm\" (UniqueName: \"kubernetes.io/projected/8905a6a4-dc65-429d-a392-7c4f396f8acd-kube-api-access-lrspm\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081498 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081379 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.081498 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.081405 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182451 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182415 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182636 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182463 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182636 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182490 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182636 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182516 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182636 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182540 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182636 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182571 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8905a6a4-dc65-429d-a392-7c4f396f8acd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182636 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182598 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8905a6a4-dc65-429d-a392-7c4f396f8acd-config-out\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182636 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182621 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-config\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182642 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182666 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-web-config\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182709 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182745 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182775 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182807 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182831 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182855 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182900 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182926 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182961 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrspm\" (UniqueName: \"kubernetes.io/projected/8905a6a4-dc65-429d-a392-7c4f396f8acd-kube-api-access-lrspm\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.182995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.182994 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.183765 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.183504 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.183765 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.183632 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.183928 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.183901 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.185799 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.185715 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8905a6a4-dc65-429d-a392-7c4f396f8acd-config-out\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.186492 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.185973 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.186492 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.186172 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.186492 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.186177 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.186492 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.186386 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.186765 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.186478 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.186765 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.186617 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8905a6a4-dc65-429d-a392-7c4f396f8acd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.186865 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.186779 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-web-config\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.187152 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.187072 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.187482 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.187454 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.187776 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.187751 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.187857 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.187800 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.188505 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.188485 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-config\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.188837 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.188815 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.189232 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.189214 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8905a6a4-dc65-429d-a392-7c4f396f8acd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.190543 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.190521 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8905a6a4-dc65-429d-a392-7c4f396f8acd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.195034 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.195017 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrspm\" (UniqueName: \"kubernetes.io/projected/8905a6a4-dc65-429d-a392-7c4f396f8acd-kube-api-access-lrspm\") pod \"prometheus-k8s-0\" (UID: \"8905a6a4-dc65-429d-a392-7c4f396f8acd\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.303456 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.303407 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:29.583249 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.583210 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 19 09:54:29.586390 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:54:29.586362 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8905a6a4_dc65_429d_a392_7c4f396f8acd.slice/crio-95d3adcbc7a2e7e2988990d166f630d0e234e9e3205817d1636690e8c5bc5eee WatchSource:0}: Error finding container 95d3adcbc7a2e7e2988990d166f630d0e234e9e3205817d1636690e8c5bc5eee: Status 404 returned error can't find the container with id 95d3adcbc7a2e7e2988990d166f630d0e234e9e3205817d1636690e8c5bc5eee Sep 19 09:54:29.943465 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.943374 2568 generic.go:356] "Generic (PLEG): container finished" podID="8905a6a4-dc65-429d-a392-7c4f396f8acd" containerID="cd00f402df8e3fdacb578c8dc611f32722cd1c862929022dcaccbe6c86df7e70" exitCode=0 Sep 19 09:54:29.943802 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.943460 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8905a6a4-dc65-429d-a392-7c4f396f8acd","Type":"ContainerDied","Data":"cd00f402df8e3fdacb578c8dc611f32722cd1c862929022dcaccbe6c86df7e70"} Sep 19 09:54:29.943802 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:29.943527 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8905a6a4-dc65-429d-a392-7c4f396f8acd","Type":"ContainerStarted","Data":"95d3adcbc7a2e7e2988990d166f630d0e234e9e3205817d1636690e8c5bc5eee"} Sep 19 09:54:30.086821 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:30.086786 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38160819-b5e0-4401-bc3c-363dcafda28d" path="/var/lib/kubelet/pods/38160819-b5e0-4401-bc3c-363dcafda28d/volumes" Sep 19 09:54:30.951899 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:30.951859 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8905a6a4-dc65-429d-a392-7c4f396f8acd","Type":"ContainerStarted","Data":"a23b5f03d00c32a535ae718b0eb53be1fc49fe3d5a0cf4ff440e153a84f3c5ce"} Sep 19 09:54:30.952336 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:30.951944 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8905a6a4-dc65-429d-a392-7c4f396f8acd","Type":"ContainerStarted","Data":"17a2b2e7fb461a72392157538a66c3872ac8191912dc6946953cf121b32911fb"} Sep 19 09:54:30.952336 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:30.951984 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8905a6a4-dc65-429d-a392-7c4f396f8acd","Type":"ContainerStarted","Data":"42fa87d9744e75599c34123aa536998d201706ac512197bd5710358cea2dde0c"} Sep 19 09:54:30.952336 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:30.952026 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8905a6a4-dc65-429d-a392-7c4f396f8acd","Type":"ContainerStarted","Data":"17e6bb6d7436d35926786195b6fc0de6a9f4b9956311b1a4b18a02bf6ac1ebf2"} Sep 19 09:54:30.952336 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:30.952067 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8905a6a4-dc65-429d-a392-7c4f396f8acd","Type":"ContainerStarted","Data":"4991f0437dba1ad770c29749d84456662bf941deb176d648c0af09251a7e746e"} Sep 19 09:54:30.952336 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:30.952101 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8905a6a4-dc65-429d-a392-7c4f396f8acd","Type":"ContainerStarted","Data":"4b04d92616caa68d325d4cf749289ebd3c54ca96c9185587f8c5ba142ddc6c67"} Sep 19 09:54:30.979773 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:30.979722 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.963181398 podStartE2EDuration="2.979704289s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.522891ms" isStatefulPod=false podCreationTimestamp="2026-09-19 09:54:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:54:30.978372764 +0000 UTC m=+177.385884433" watchObservedRunningTime="2026-09-19 09:54:30.979704289 +0000 UTC m=+177.387215959" Sep 19 09:54:31.333313 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:31.333282 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:31.434388 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:31.434357 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:34.303733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:34.303627 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:54:36.506289 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:36.506250 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:36.607889 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:36.607852 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:36.707385 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:36.707354 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:37.299176 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:37.299126 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:37.400088 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:37.400054 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:54:37.501793 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:54:37.501755 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:11.929044 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:11.929011 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:11.981651 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:11.981608 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:18.585993 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:18.585956 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:18.690243 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:18.690198 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:29.303903 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:29.303863 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:55:29.320090 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:29.320061 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:55:30.195891 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:30.195859 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 19 09:55:38.448503 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:38.448458 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:39.581572 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:39.581532 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:40.200535 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:40.200501 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:41.148774 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:41.148738 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:43.809510 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:43.809470 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:43.915800 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:43.915766 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:44.807012 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:44.806973 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:44.951323 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:44.951279 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:46.091425 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:46.091378 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:46.188645 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:46.188611 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:48.307425 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:48.307394 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:48.544965 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:48.544929 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:48.856329 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:48.856291 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8"] Sep 19 09:55:48.858925 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:48.858904 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8" Sep 19 09:55:48.861755 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:48.861735 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8n5gq\"" Sep 19 09:55:48.861882 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:48.861776 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Sep 19 09:55:48.862370 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:48.862354 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Sep 19 09:55:48.870301 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:48.870278 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8"] Sep 19 09:55:48.968012 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:48.967974 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vdsp\" (UniqueName: \"kubernetes.io/projected/23626ce0-fa44-416c-9197-05538e1229c6-kube-api-access-8vdsp\") pod \"cert-manager-cainjector-559cdd8c87-wkqv8\" (UID: \"23626ce0-fa44-416c-9197-05538e1229c6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8" Sep 19 09:55:48.968206 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:48.968046 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23626ce0-fa44-416c-9197-05538e1229c6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wkqv8\" (UID: \"23626ce0-fa44-416c-9197-05538e1229c6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8" Sep 19 09:55:49.068499 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:49.068460 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23626ce0-fa44-416c-9197-05538e1229c6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wkqv8\" (UID: \"23626ce0-fa44-416c-9197-05538e1229c6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8" Sep 19 09:55:49.068677 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:49.068531 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vdsp\" (UniqueName: \"kubernetes.io/projected/23626ce0-fa44-416c-9197-05538e1229c6-kube-api-access-8vdsp\") pod \"cert-manager-cainjector-559cdd8c87-wkqv8\" (UID: \"23626ce0-fa44-416c-9197-05538e1229c6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8" Sep 19 09:55:49.076930 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:49.076905 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23626ce0-fa44-416c-9197-05538e1229c6-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wkqv8\" (UID: \"23626ce0-fa44-416c-9197-05538e1229c6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8" Sep 19 09:55:49.077098 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:49.077028 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vdsp\" (UniqueName: \"kubernetes.io/projected/23626ce0-fa44-416c-9197-05538e1229c6-kube-api-access-8vdsp\") pod \"cert-manager-cainjector-559cdd8c87-wkqv8\" (UID: \"23626ce0-fa44-416c-9197-05538e1229c6\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8" Sep 19 09:55:49.167985 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:49.167891 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8" Sep 19 09:55:49.438586 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:49.438564 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8"] Sep 19 09:55:49.443986 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:55:49.443950 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23626ce0_fa44_416c_9197_05538e1229c6.slice/crio-594e3ae09c912a6997782799b1daac8f6bdf87a637e206bbea69dad8e18c02b2 WatchSource:0}: Error finding container 594e3ae09c912a6997782799b1daac8f6bdf87a637e206bbea69dad8e18c02b2: Status 404 returned error can't find the container with id 594e3ae09c912a6997782799b1daac8f6bdf87a637e206bbea69dad8e18c02b2 Sep 19 09:55:50.259118 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:50.259079 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8" event={"ID":"23626ce0-fa44-416c-9197-05538e1229c6","Type":"ContainerStarted","Data":"594e3ae09c912a6997782799b1daac8f6bdf87a637e206bbea69dad8e18c02b2"} Sep 19 09:55:50.621723 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:50.621686 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:50.677615 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:50.677579 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:53.272334 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:53.272302 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8" event={"ID":"23626ce0-fa44-416c-9197-05538e1229c6","Type":"ContainerStarted","Data":"fb40f11693ae476aff057f43a8b16a83be68914a38fdf2b87985eee02cbccf85"} Sep 19 09:55:53.292558 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:53.292505 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wkqv8" podStartSLOduration=2.483964394 podStartE2EDuration="5.292489067s" totalImagesPullingTime="2.808524673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:55:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:55:53.291280945 +0000 UTC m=+259.698792629" watchObservedRunningTime="2026-09-19 09:55:53.292489067 +0000 UTC m=+259.700000735" Sep 19 09:55:54.129241 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:54.129206 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:54.193583 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:54.193550 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:55:56.515914 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:55:56.515883 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:02.025106 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:02.025063 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:02.122095 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:02.122064 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:08.437928 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:08.437852 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:08.541460 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:08.541429 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:10.254379 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:10.254340 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:22.914782 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:22.914751 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:23.014106 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:23.014067 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:23.560547 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:23.560514 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:23.779259 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:23.779228 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:23.879414 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:23.879378 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:23.970266 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:23.970233 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm"] Sep 19 09:56:23.977211 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:23.977181 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:23.980410 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:23.980387 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 19 09:56:23.980539 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:23.980388 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 19 09:56:23.981199 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:23.981177 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 19 09:56:23.981296 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:23.981177 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-n2xq2\"" Sep 19 09:56:23.981296 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:23.981183 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 19 09:56:23.998566 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:23.998541 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm"] Sep 19 09:56:24.058907 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.058876 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e979c06f-682d-4d31-a021-85f46c825383-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.058907 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.058915 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e979c06f-682d-4d31-a021-85f46c825383-tmp\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.059117 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.058935 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e979c06f-682d-4d31-a021-85f46c825383-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.059117 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.059015 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g9db\" (UniqueName: \"kubernetes.io/projected/e979c06f-682d-4d31-a021-85f46c825383-kube-api-access-5g9db\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.059117 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.059070 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e979c06f-682d-4d31-a021-85f46c825383-manifests\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.135111 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.135034 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:24.160538 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.160501 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e979c06f-682d-4d31-a021-85f46c825383-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.160724 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.160546 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e979c06f-682d-4d31-a021-85f46c825383-tmp\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.160724 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.160675 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e979c06f-682d-4d31-a021-85f46c825383-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.160834 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.160773 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g9db\" (UniqueName: \"kubernetes.io/projected/e979c06f-682d-4d31-a021-85f46c825383-kube-api-access-5g9db\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.160892 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.160855 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e979c06f-682d-4d31-a021-85f46c825383-manifests\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.161252 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.161226 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e979c06f-682d-4d31-a021-85f46c825383-manifests\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.162965 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.162943 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e979c06f-682d-4d31-a021-85f46c825383-tmp\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.163213 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.163191 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e979c06f-682d-4d31-a021-85f46c825383-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.163274 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.163250 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e979c06f-682d-4d31-a021-85f46c825383-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.173070 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.173031 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g9db\" (UniqueName: \"kubernetes.io/projected/e979c06f-682d-4d31-a021-85f46c825383-kube-api-access-5g9db\") pod \"opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm\" (UID: \"e979c06f-682d-4d31-a021-85f46c825383\") " pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.233093 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.233061 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:24.287240 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.287204 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:24.333204 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.333171 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:24.434970 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.434870 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:24.537617 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.537584 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:24.603835 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.603811 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm"] Sep 19 09:56:24.605537 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:56:24.605496 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode979c06f_682d_4d31_a021_85f46c825383.slice/crio-47f2689685c92ea0f07d65420229784e18390bf706781b8592a28f6fb3277301 WatchSource:0}: Error finding container 47f2689685c92ea0f07d65420229784e18390bf706781b8592a28f6fb3277301: Status 404 returned error can't find the container with id 47f2689685c92ea0f07d65420229784e18390bf706781b8592a28f6fb3277301 Sep 19 09:56:24.656633 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.656594 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84d7567998-v2cts"] Sep 19 09:56:24.660056 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.660031 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.663005 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.662980 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Sep 19 09:56:24.663249 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.663236 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9kbqf\"" Sep 19 09:56:24.663498 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.663481 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Sep 19 09:56:24.664227 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.664210 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Sep 19 09:56:24.664314 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.664210 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Sep 19 09:56:24.667715 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.667695 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Sep 19 09:56:24.669830 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.669810 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:24.715887 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.715805 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84d7567998-v2cts"] Sep 19 09:56:24.757024 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.756978 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:24.767022 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.766992 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2951106a-dc2f-4d40-add3-7bd5ec37906d-cert\") pod \"lws-controller-manager-84d7567998-v2cts\" (UID: \"2951106a-dc2f-4d40-add3-7bd5ec37906d\") " pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.767237 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.767036 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2951106a-dc2f-4d40-add3-7bd5ec37906d-metrics-cert\") pod \"lws-controller-manager-84d7567998-v2cts\" (UID: \"2951106a-dc2f-4d40-add3-7bd5ec37906d\") " pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.767237 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.767163 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tls2\" (UniqueName: \"kubernetes.io/projected/2951106a-dc2f-4d40-add3-7bd5ec37906d-kube-api-access-7tls2\") pod \"lws-controller-manager-84d7567998-v2cts\" (UID: \"2951106a-dc2f-4d40-add3-7bd5ec37906d\") " pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.767237 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.767220 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2951106a-dc2f-4d40-add3-7bd5ec37906d-manager-config\") pod \"lws-controller-manager-84d7567998-v2cts\" (UID: \"2951106a-dc2f-4d40-add3-7bd5ec37906d\") " pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.852046 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.852014 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:24.868147 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.868093 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2951106a-dc2f-4d40-add3-7bd5ec37906d-cert\") pod \"lws-controller-manager-84d7567998-v2cts\" (UID: \"2951106a-dc2f-4d40-add3-7bd5ec37906d\") " pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.868147 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.868148 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2951106a-dc2f-4d40-add3-7bd5ec37906d-metrics-cert\") pod \"lws-controller-manager-84d7567998-v2cts\" (UID: \"2951106a-dc2f-4d40-add3-7bd5ec37906d\") " pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.868390 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.868204 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tls2\" (UniqueName: \"kubernetes.io/projected/2951106a-dc2f-4d40-add3-7bd5ec37906d-kube-api-access-7tls2\") pod \"lws-controller-manager-84d7567998-v2cts\" (UID: \"2951106a-dc2f-4d40-add3-7bd5ec37906d\") " pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.868390 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.868237 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2951106a-dc2f-4d40-add3-7bd5ec37906d-manager-config\") pod \"lws-controller-manager-84d7567998-v2cts\" (UID: \"2951106a-dc2f-4d40-add3-7bd5ec37906d\") " pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.868903 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.868872 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2951106a-dc2f-4d40-add3-7bd5ec37906d-manager-config\") pod \"lws-controller-manager-84d7567998-v2cts\" (UID: \"2951106a-dc2f-4d40-add3-7bd5ec37906d\") " pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.871663 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.871638 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2951106a-dc2f-4d40-add3-7bd5ec37906d-cert\") pod \"lws-controller-manager-84d7567998-v2cts\" (UID: \"2951106a-dc2f-4d40-add3-7bd5ec37906d\") " pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.871815 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.871658 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2951106a-dc2f-4d40-add3-7bd5ec37906d-metrics-cert\") pod \"lws-controller-manager-84d7567998-v2cts\" (UID: \"2951106a-dc2f-4d40-add3-7bd5ec37906d\") " pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.879193 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.879164 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tls2\" (UniqueName: \"kubernetes.io/projected/2951106a-dc2f-4d40-add3-7bd5ec37906d-kube-api-access-7tls2\") pod \"lws-controller-manager-84d7567998-v2cts\" (UID: \"2951106a-dc2f-4d40-add3-7bd5ec37906d\") " pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:24.969754 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:24.969660 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:25.255405 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:25.254981 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84d7567998-v2cts"] Sep 19 09:56:25.258258 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:56:25.258224 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2951106a_dc2f_4d40_add3_7bd5ec37906d.slice/crio-b48e8eb3b4193a3dc5cabdeb7927acf49bc940eb98fbf25e75b0267a988234a3 WatchSource:0}: Error finding container b48e8eb3b4193a3dc5cabdeb7927acf49bc940eb98fbf25e75b0267a988234a3: Status 404 returned error can't find the container with id b48e8eb3b4193a3dc5cabdeb7927acf49bc940eb98fbf25e75b0267a988234a3 Sep 19 09:56:25.403672 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:25.403589 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" event={"ID":"e979c06f-682d-4d31-a021-85f46c825383","Type":"ContainerStarted","Data":"47f2689685c92ea0f07d65420229784e18390bf706781b8592a28f6fb3277301"} Sep 19 09:56:25.405218 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:25.405179 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" event={"ID":"2951106a-dc2f-4d40-add3-7bd5ec37906d","Type":"ContainerStarted","Data":"b48e8eb3b4193a3dc5cabdeb7927acf49bc940eb98fbf25e75b0267a988234a3"} Sep 19 09:56:29.433804 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:29.433767 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" event={"ID":"2951106a-dc2f-4d40-add3-7bd5ec37906d","Type":"ContainerStarted","Data":"ae3fb862464a8aeaa0b9ab8695c640f803e5bd2a3447152455f7a30d20f1a1ab"} Sep 19 09:56:29.434304 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:29.433828 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:29.435524 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:29.435498 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" event={"ID":"e979c06f-682d-4d31-a021-85f46c825383","Type":"ContainerStarted","Data":"8f5cb62a9b3bff0de4ce2099d2a28ad626c08bd010d03b7f416719b9706a059a"} Sep 19 09:56:29.451506 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:29.451455 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" podStartSLOduration=1.9825130290000001 podStartE2EDuration="5.451439738s" totalImagesPullingTime="3.468926709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:56:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:56:29.450338643 +0000 UTC m=+295.857850311" watchObservedRunningTime="2026-09-19 09:56:29.451439738 +0000 UTC m=+295.858951407" Sep 19 09:56:30.441216 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:30.441180 2568 generic.go:356] "Generic (PLEG): container finished" podID="e979c06f-682d-4d31-a021-85f46c825383" containerID="8f5cb62a9b3bff0de4ce2099d2a28ad626c08bd010d03b7f416719b9706a059a" exitCode=0 Sep 19 09:56:30.441717 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:30.441281 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" event={"ID":"e979c06f-682d-4d31-a021-85f46c825383","Type":"ContainerDied","Data":"8f5cb62a9b3bff0de4ce2099d2a28ad626c08bd010d03b7f416719b9706a059a"} Sep 19 09:56:31.448234 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:31.448197 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" event={"ID":"e979c06f-682d-4d31-a021-85f46c825383","Type":"ContainerStarted","Data":"0916c8203d5592d15a3abffdc7c913fe1a6f4278e734c186b7297115dcbf39df"} Sep 19 09:56:31.448601 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:31.448280 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:31.469741 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:31.469686 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" podStartSLOduration=3.086457799 podStartE2EDuration="8.469660785s" totalImagesPullingTime="4.348497255s" totalInitContainerRuntime="1.034705731s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:56:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:56:31.468003886 +0000 UTC m=+297.875515596" watchObservedRunningTime="2026-09-19 09:56:31.469660785 +0000 UTC m=+297.877172457" Sep 19 09:56:33.198934 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:33.198906 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:34.010456 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:34.010428 2568 kubelet.go:1648] "Image garbage collection succeeded" Sep 19 09:56:34.215661 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:34.215629 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:34.558946 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:34.558911 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:34.659638 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:34.659602 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:40.443993 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:40.443956 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84d7567998-v2cts" Sep 19 09:56:42.455164 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:42.455108 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm" Sep 19 09:56:44.237335 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.237294 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:44.333352 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.333316 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:44.504605 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.504509 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:44.543230 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.543195 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz"] Sep 19 09:56:44.556473 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.556444 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" Sep 19 09:56:44.560885 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.560858 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 19 09:56:44.561033 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.560906 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 19 09:56:44.561033 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.560935 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 19 09:56:44.561816 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.561792 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 19 09:56:44.561939 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.561871 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7qv9r\"" Sep 19 09:56:44.567786 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.567766 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz"] Sep 19 09:56:44.645733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.645701 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e40e2e2-b805-47b0-88e5-9fcbdd3a843c-tls-certs\") pod \"kube-auth-proxy-65d7d84b5d-4j8rz\" (UID: \"3e40e2e2-b805-47b0-88e5-9fcbdd3a843c\") " pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" Sep 19 09:56:44.645911 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.645762 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e40e2e2-b805-47b0-88e5-9fcbdd3a843c-tmp\") pod \"kube-auth-proxy-65d7d84b5d-4j8rz\" (UID: \"3e40e2e2-b805-47b0-88e5-9fcbdd3a843c\") " pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" Sep 19 09:56:44.645911 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.645808 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg69r\" (UniqueName: \"kubernetes.io/projected/3e40e2e2-b805-47b0-88e5-9fcbdd3a843c-kube-api-access-wg69r\") pod \"kube-auth-proxy-65d7d84b5d-4j8rz\" (UID: \"3e40e2e2-b805-47b0-88e5-9fcbdd3a843c\") " pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" Sep 19 09:56:44.646090 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.646068 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:44.746388 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.746356 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg69r\" (UniqueName: \"kubernetes.io/projected/3e40e2e2-b805-47b0-88e5-9fcbdd3a843c-kube-api-access-wg69r\") pod \"kube-auth-proxy-65d7d84b5d-4j8rz\" (UID: \"3e40e2e2-b805-47b0-88e5-9fcbdd3a843c\") " pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" Sep 19 09:56:44.746572 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.746428 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e40e2e2-b805-47b0-88e5-9fcbdd3a843c-tls-certs\") pod \"kube-auth-proxy-65d7d84b5d-4j8rz\" (UID: \"3e40e2e2-b805-47b0-88e5-9fcbdd3a843c\") " pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" Sep 19 09:56:44.746572 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.746498 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e40e2e2-b805-47b0-88e5-9fcbdd3a843c-tmp\") pod \"kube-auth-proxy-65d7d84b5d-4j8rz\" (UID: \"3e40e2e2-b805-47b0-88e5-9fcbdd3a843c\") " pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" Sep 19 09:56:44.748860 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.748838 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e40e2e2-b805-47b0-88e5-9fcbdd3a843c-tmp\") pod \"kube-auth-proxy-65d7d84b5d-4j8rz\" (UID: \"3e40e2e2-b805-47b0-88e5-9fcbdd3a843c\") " pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" Sep 19 09:56:44.748986 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.748970 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3e40e2e2-b805-47b0-88e5-9fcbdd3a843c-tls-certs\") pod \"kube-auth-proxy-65d7d84b5d-4j8rz\" (UID: \"3e40e2e2-b805-47b0-88e5-9fcbdd3a843c\") " pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" Sep 19 09:56:44.764759 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.764697 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg69r\" (UniqueName: \"kubernetes.io/projected/3e40e2e2-b805-47b0-88e5-9fcbdd3a843c-kube-api-access-wg69r\") pod \"kube-auth-proxy-65d7d84b5d-4j8rz\" (UID: \"3e40e2e2-b805-47b0-88e5-9fcbdd3a843c\") " pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" Sep 19 09:56:44.866462 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:44.866426 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" Sep 19 09:56:45.142090 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:45.141902 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz"] Sep 19 09:56:45.144822 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:56:45.144795 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e40e2e2_b805_47b0_88e5_9fcbdd3a843c.slice/crio-9986f5446685f16488b0628276d5b5b3cb735aab5664f154c29958eaf9615727 WatchSource:0}: Error finding container 9986f5446685f16488b0628276d5b5b3cb735aab5664f154c29958eaf9615727: Status 404 returned error can't find the container with id 9986f5446685f16488b0628276d5b5b3cb735aab5664f154c29958eaf9615727 Sep 19 09:56:45.146646 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:45.146633 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 09:56:45.511555 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:45.511468 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" event={"ID":"3e40e2e2-b805-47b0-88e5-9fcbdd3a843c","Type":"ContainerStarted","Data":"9986f5446685f16488b0628276d5b5b3cb735aab5664f154c29958eaf9615727"} Sep 19 09:56:45.812747 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:45.812597 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:45.873714 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:45.873656 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:46.765007 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:46.764975 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:48.527612 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:48.527577 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" event={"ID":"3e40e2e2-b805-47b0-88e5-9fcbdd3a843c","Type":"ContainerStarted","Data":"114870cc7a199a8a2e55291a925818987df4f24a68c9abecd11b9360afd53cda"} Sep 19 09:56:48.545097 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:48.545028 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65d7d84b5d-4j8rz" podStartSLOduration=1.747850359 podStartE2EDuration="4.545013026s" totalImagesPullingTime="2.797162667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:56:48.544444313 +0000 UTC m=+314.951956009" watchObservedRunningTime="2026-09-19 09:56:48.545013026 +0000 UTC m=+314.952524714" Sep 19 09:56:50.431442 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:50.431398 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:50.531289 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:50.531254 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:51.116110 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:51.116076 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:51.217729 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:51.217692 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:51.315222 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:51.315190 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:55.496211 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:55.496169 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:55.554131 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:55.554093 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:55.657429 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:55.657388 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:55.898184 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:55.898149 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:55.985417 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:55.985376 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:56.508874 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:56.508831 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:58.796995 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:58.796962 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:56:58.852658 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:56:58.852628 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:04.371408 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:04.371358 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:04.443776 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:04.443746 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:04.499423 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:04.499381 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:04.619891 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:04.619854 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:04.727189 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:04.727076 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:04.826097 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:04.826039 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:04.896722 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:04.896683 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:52.706422 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.706339 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d6556bd68-pt9sg"] Sep 19 09:57:52.708736 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.708714 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.712099 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.712063 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 19 09:57:52.712251 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.712102 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 19 09:57:52.712251 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.712125 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-b6mb7\"" Sep 19 09:57:52.712251 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.712113 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 19 09:57:52.712251 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.712178 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 19 09:57:52.712421 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.712312 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 19 09:57:52.717209 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.717191 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 19 09:57:52.720922 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.720900 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d6556bd68-pt9sg"] Sep 19 09:57:52.732510 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.732483 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e08ca42f-5053-4609-b195-53bb7ce1925b-service-ca\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.732663 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.732523 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e08ca42f-5053-4609-b195-53bb7ce1925b-console-config\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.732663 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.732542 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e08ca42f-5053-4609-b195-53bb7ce1925b-oauth-serving-cert\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.732663 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.732594 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxbvj\" (UniqueName: \"kubernetes.io/projected/e08ca42f-5053-4609-b195-53bb7ce1925b-kube-api-access-pxbvj\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.732663 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.732640 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e08ca42f-5053-4609-b195-53bb7ce1925b-console-oauth-config\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.732857 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.732715 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e08ca42f-5053-4609-b195-53bb7ce1925b-trusted-ca-bundle\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.732857 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.732753 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e08ca42f-5053-4609-b195-53bb7ce1925b-console-serving-cert\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.762488 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.762447 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:52.818629 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.818596 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:52.833743 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.833715 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e08ca42f-5053-4609-b195-53bb7ce1925b-service-ca\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.833890 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.833754 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e08ca42f-5053-4609-b195-53bb7ce1925b-console-config\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.833890 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.833771 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e08ca42f-5053-4609-b195-53bb7ce1925b-oauth-serving-cert\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.833994 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.833886 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxbvj\" (UniqueName: \"kubernetes.io/projected/e08ca42f-5053-4609-b195-53bb7ce1925b-kube-api-access-pxbvj\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.833994 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.833938 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e08ca42f-5053-4609-b195-53bb7ce1925b-console-oauth-config\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.833994 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.833976 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e08ca42f-5053-4609-b195-53bb7ce1925b-trusted-ca-bundle\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.834227 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.833996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e08ca42f-5053-4609-b195-53bb7ce1925b-console-serving-cert\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.834553 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.834523 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e08ca42f-5053-4609-b195-53bb7ce1925b-console-config\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.834657 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.834612 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e08ca42f-5053-4609-b195-53bb7ce1925b-oauth-serving-cert\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.834657 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.834633 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e08ca42f-5053-4609-b195-53bb7ce1925b-service-ca\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.834848 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.834831 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e08ca42f-5053-4609-b195-53bb7ce1925b-trusted-ca-bundle\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.837185 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.837155 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e08ca42f-5053-4609-b195-53bb7ce1925b-console-oauth-config\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.837267 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.837163 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e08ca42f-5053-4609-b195-53bb7ce1925b-console-serving-cert\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.843213 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.843193 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxbvj\" (UniqueName: \"kubernetes.io/projected/e08ca42f-5053-4609-b195-53bb7ce1925b-kube-api-access-pxbvj\") pod \"console-5d6556bd68-pt9sg\" (UID: \"e08ca42f-5053-4609-b195-53bb7ce1925b\") " pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:52.908289 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:52.908246 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:53.019192 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:53.019081 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:57:53.294953 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:53.294928 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d6556bd68-pt9sg"] Sep 19 09:57:53.297689 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:57:53.297659 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode08ca42f_5053_4609_b195_53bb7ce1925b.slice/crio-50eff347451511ac7f663365c3ca27401c3ad6ba442d858d1d8970349dfd621b WatchSource:0}: Error finding container 50eff347451511ac7f663365c3ca27401c3ad6ba442d858d1d8970349dfd621b: Status 404 returned error can't find the container with id 50eff347451511ac7f663365c3ca27401c3ad6ba442d858d1d8970349dfd621b Sep 19 09:57:53.510803 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:53.510771 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:53.610207 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:53.610161 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:53.815538 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:53.815497 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d6556bd68-pt9sg" event={"ID":"e08ca42f-5053-4609-b195-53bb7ce1925b","Type":"ContainerStarted","Data":"ff03ba9437fd4302ab834fcec357baef8e63908299a66f0513920b6785877cba"} Sep 19 09:57:53.815538 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:53.815541 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d6556bd68-pt9sg" event={"ID":"e08ca42f-5053-4609-b195-53bb7ce1925b","Type":"ContainerStarted","Data":"50eff347451511ac7f663365c3ca27401c3ad6ba442d858d1d8970349dfd621b"} Sep 19 09:57:53.835819 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:53.835764 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d6556bd68-pt9sg" podStartSLOduration=1.835745642 podStartE2EDuration="1.835745642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:57:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:57:53.833893108 +0000 UTC m=+380.241404800" watchObservedRunningTime="2026-09-19 09:57:53.835745642 +0000 UTC m=+380.243257311" Sep 19 09:57:56.313928 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:56.313895 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:57:56.413494 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:57:56.413457 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:02.652509 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:02.652474 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:02.708917 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:02.708874 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:03.019883 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:03.019786 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:58:03.019883 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:03.019855 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:58:03.024713 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:03.024688 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:58:03.866374 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:03.866341 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d6556bd68-pt9sg" Sep 19 09:58:04.026458 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:04.026416 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:04.127347 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:04.127250 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:04.188399 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:04.188364 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:20.922587 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:20.922555 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:21.721357 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:21.721319 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:24.757624 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:24.757590 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:24.865116 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:24.865080 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:24.971856 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:24.971814 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:26.758207 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:26.758171 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:26.932985 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:26.932946 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:27.514761 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:27.514727 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:27.757322 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:27.757284 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:27.852374 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:27.852342 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:27.953461 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:27.953417 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:28.062945 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:28.062913 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:31.125912 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:31.125874 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:32.158089 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:32.158054 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:34.130931 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:34.130897 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:34.231460 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:34.231416 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:34.520456 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:34.520353 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:34.573089 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:34.573049 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:37.839460 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:37.839389 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:37.992319 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:37.992288 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:38.124550 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:38.124460 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:38.225466 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:38.225432 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:42.475461 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:42.475421 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:42.573410 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:42.573372 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:42.632609 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:42.632585 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:45.329602 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:45.329554 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:45.429937 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:45.429907 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:47.427827 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.427792 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc"] Sep 19 09:58:47.431593 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.431575 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:47.434559 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.434531 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 19 09:58:47.434672 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.434610 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 19 09:58:47.434672 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.434627 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-ln5h6\"" Sep 19 09:58:47.448772 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.448749 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc"] Sep 19 09:58:47.520166 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.520095 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpxzh\" (UniqueName: \"kubernetes.io/projected/a2933953-2aba-4b93-9650-b48f38270fbc-kube-api-access-gpxzh\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-jf7sc\" (UID: \"a2933953-2aba-4b93-9650-b48f38270fbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:47.520166 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.520157 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2933953-2aba-4b93-9650-b48f38270fbc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-jf7sc\" (UID: \"a2933953-2aba-4b93-9650-b48f38270fbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:47.621218 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.621166 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpxzh\" (UniqueName: \"kubernetes.io/projected/a2933953-2aba-4b93-9650-b48f38270fbc-kube-api-access-gpxzh\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-jf7sc\" (UID: \"a2933953-2aba-4b93-9650-b48f38270fbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:47.621218 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.621227 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2933953-2aba-4b93-9650-b48f38270fbc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-jf7sc\" (UID: \"a2933953-2aba-4b93-9650-b48f38270fbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:47.621631 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.621613 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2933953-2aba-4b93-9650-b48f38270fbc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-jf7sc\" (UID: \"a2933953-2aba-4b93-9650-b48f38270fbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:47.641732 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.641707 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpxzh\" (UniqueName: \"kubernetes.io/projected/a2933953-2aba-4b93-9650-b48f38270fbc-kube-api-access-gpxzh\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-jf7sc\" (UID: \"a2933953-2aba-4b93-9650-b48f38270fbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:47.741922 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:47.741840 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:48.018364 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.018281 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc"] Sep 19 09:58:48.021874 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:58:48.021847 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2933953_2aba_4b93_9650_b48f38270fbc.slice/crio-f7d5025c47568b4013396ea4248209ca31f0bd3c567afd4b52626d3f815f635c WatchSource:0}: Error finding container f7d5025c47568b4013396ea4248209ca31f0bd3c567afd4b52626d3f815f635c: Status 404 returned error can't find the container with id f7d5025c47568b4013396ea4248209ca31f0bd3c567afd4b52626d3f815f635c Sep 19 09:58:48.062211 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.062172 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" event={"ID":"a2933953-2aba-4b93-9650-b48f38270fbc","Type":"ContainerStarted","Data":"f7d5025c47568b4013396ea4248209ca31f0bd3c567afd4b52626d3f815f635c"} Sep 19 09:58:48.456633 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.456594 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc"] Sep 19 09:58:48.460367 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.460340 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc"] Sep 19 09:58:48.478468 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.478441 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g"] Sep 19 09:58:48.483040 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.483017 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" Sep 19 09:58:48.495053 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.495031 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g"] Sep 19 09:58:48.527707 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.527675 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f002f41d-3f45-4277-8be4-3b9875d78a24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-w5l7g\" (UID: \"f002f41d-3f45-4277-8be4-3b9875d78a24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" Sep 19 09:58:48.527899 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.527712 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7btx\" (UniqueName: \"kubernetes.io/projected/f002f41d-3f45-4277-8be4-3b9875d78a24-kube-api-access-l7btx\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-w5l7g\" (UID: \"f002f41d-3f45-4277-8be4-3b9875d78a24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" Sep 19 09:58:48.629233 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.629196 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f002f41d-3f45-4277-8be4-3b9875d78a24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-w5l7g\" (UID: \"f002f41d-3f45-4277-8be4-3b9875d78a24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" Sep 19 09:58:48.629233 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.629234 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7btx\" (UniqueName: \"kubernetes.io/projected/f002f41d-3f45-4277-8be4-3b9875d78a24-kube-api-access-l7btx\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-w5l7g\" (UID: \"f002f41d-3f45-4277-8be4-3b9875d78a24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" Sep 19 09:58:48.629582 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.629561 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f002f41d-3f45-4277-8be4-3b9875d78a24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-w5l7g\" (UID: \"f002f41d-3f45-4277-8be4-3b9875d78a24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" Sep 19 09:58:48.637427 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.637402 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7btx\" (UniqueName: \"kubernetes.io/projected/f002f41d-3f45-4277-8be4-3b9875d78a24-kube-api-access-l7btx\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-w5l7g\" (UID: \"f002f41d-3f45-4277-8be4-3b9875d78a24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" Sep 19 09:58:48.793652 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:48.793572 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" Sep 19 09:58:49.034734 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:49.034699 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:49.077338 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:49.077315 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g"] Sep 19 09:58:49.079810 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:58:49.079785 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf002f41d_3f45_4277_8be4_3b9875d78a24.slice/crio-fd8bbae17b4da52f4ffc760565992f5cbae073bf211ad4b3335e3e59db1cddc6 WatchSource:0}: Error finding container fd8bbae17b4da52f4ffc760565992f5cbae073bf211ad4b3335e3e59db1cddc6: Status 404 returned error can't find the container with id fd8bbae17b4da52f4ffc760565992f5cbae073bf211ad4b3335e3e59db1cddc6 Sep 19 09:58:49.124659 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:49.124623 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:50.076698 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:50.076599 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" event={"ID":"f002f41d-3f45-4277-8be4-3b9875d78a24","Type":"ContainerStarted","Data":"fd8bbae17b4da52f4ffc760565992f5cbae073bf211ad4b3335e3e59db1cddc6"} Sep 19 09:58:54.103699 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.103624 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" podUID="a2933953-2aba-4b93-9650-b48f38270fbc" containerName="manager" containerID="cri-o://7adea9fcda66fa547acc0d7c8744bd6725bdc073cf6970140d4856757ed379e8" gracePeriod=2 Sep 19 09:58:54.105683 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.105645 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" event={"ID":"f002f41d-3f45-4277-8be4-3b9875d78a24","Type":"ContainerStarted","Data":"65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c"} Sep 19 09:58:54.105824 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:58:54.105776 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-jf7sc\" is forbidden: User \"system:node:ip-10-0-139-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-28.ec2.internal' and this object" podUID="a2933953-2aba-4b93-9650-b48f38270fbc" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:54.105824 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.105804 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" Sep 19 09:58:54.107801 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:58:54.107779 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-jf7sc\" is forbidden: User \"system:node:ip-10-0-139-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-28.ec2.internal' and this object" podUID="a2933953-2aba-4b93-9650-b48f38270fbc" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:54.128293 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.128248 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" podStartSLOduration=1.615389331 podStartE2EDuration="6.128233675s" totalImagesPullingTime="4.512844344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:58:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:58:54.125522671 +0000 UTC m=+440.533034341" watchObservedRunningTime="2026-09-19 09:58:54.128233675 +0000 UTC m=+440.535745343" Sep 19 09:58:54.347866 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.347841 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:54.349860 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:58:54.349839 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-jf7sc\" is forbidden: User \"system:node:ip-10-0-139-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-28.ec2.internal' and this object" podUID="a2933953-2aba-4b93-9650-b48f38270fbc" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:54.485938 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.485864 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2933953-2aba-4b93-9650-b48f38270fbc-extensions-socket-volume\") pod \"a2933953-2aba-4b93-9650-b48f38270fbc\" (UID: \"a2933953-2aba-4b93-9650-b48f38270fbc\") " Sep 19 09:58:54.485938 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.485908 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpxzh\" (UniqueName: \"kubernetes.io/projected/a2933953-2aba-4b93-9650-b48f38270fbc-kube-api-access-gpxzh\") pod \"a2933953-2aba-4b93-9650-b48f38270fbc\" (UID: \"a2933953-2aba-4b93-9650-b48f38270fbc\") " Sep 19 09:58:54.486155 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.486114 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2933953-2aba-4b93-9650-b48f38270fbc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a2933953-2aba-4b93-9650-b48f38270fbc" (UID: "a2933953-2aba-4b93-9650-b48f38270fbc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:58:54.488411 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.488383 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2933953-2aba-4b93-9650-b48f38270fbc-kube-api-access-gpxzh" (OuterVolumeSpecName: "kube-api-access-gpxzh") pod "a2933953-2aba-4b93-9650-b48f38270fbc" (UID: "a2933953-2aba-4b93-9650-b48f38270fbc"). InnerVolumeSpecName "kube-api-access-gpxzh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:58:54.506594 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.506555 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:54.560716 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.560672 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:54.586941 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.586909 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2933953-2aba-4b93-9650-b48f38270fbc-extensions-socket-volume\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:58:54.586941 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:54.586934 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpxzh\" (UniqueName: \"kubernetes.io/projected/a2933953-2aba-4b93-9650-b48f38270fbc-kube-api-access-gpxzh\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:58:55.107158 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:55.107047 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:55.112702 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:55.112678 2568 generic.go:356] "Generic (PLEG): container finished" podID="a2933953-2aba-4b93-9650-b48f38270fbc" containerID="7adea9fcda66fa547acc0d7c8744bd6725bdc073cf6970140d4856757ed379e8" exitCode=2 Sep 19 09:58:55.112840 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:55.112726 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:55.112840 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:55.112778 2568 scope.go:117] "RemoveContainer" containerID="7adea9fcda66fa547acc0d7c8744bd6725bdc073cf6970140d4856757ed379e8" Sep 19 09:58:55.115174 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:58:55.115148 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-jf7sc\" is forbidden: User \"system:node:ip-10-0-139-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-28.ec2.internal' and this object" podUID="a2933953-2aba-4b93-9650-b48f38270fbc" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:55.123419 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:58:55.123354 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-jf7sc\" is forbidden: User \"system:node:ip-10-0-139-28.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-28.ec2.internal' and this object" podUID="a2933953-2aba-4b93-9650-b48f38270fbc" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-jf7sc" Sep 19 09:58:55.123523 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:55.123499 2568 scope.go:117] "RemoveContainer" containerID="7adea9fcda66fa547acc0d7c8744bd6725bdc073cf6970140d4856757ed379e8" Sep 19 09:58:55.124158 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:58:55.124115 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7adea9fcda66fa547acc0d7c8744bd6725bdc073cf6970140d4856757ed379e8\": container with ID starting with 7adea9fcda66fa547acc0d7c8744bd6725bdc073cf6970140d4856757ed379e8 not found: ID does not exist" containerID="7adea9fcda66fa547acc0d7c8744bd6725bdc073cf6970140d4856757ed379e8" Sep 19 09:58:55.124248 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:55.124167 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7adea9fcda66fa547acc0d7c8744bd6725bdc073cf6970140d4856757ed379e8"} err="failed to get container status \"7adea9fcda66fa547acc0d7c8744bd6725bdc073cf6970140d4856757ed379e8\": rpc error: code = NotFound desc = could not find container \"7adea9fcda66fa547acc0d7c8744bd6725bdc073cf6970140d4856757ed379e8\": container with ID starting with 7adea9fcda66fa547acc0d7c8744bd6725bdc073cf6970140d4856757ed379e8 not found: ID does not exist" Sep 19 09:58:55.206894 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:55.206865 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:56.085445 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:56.085414 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2933953-2aba-4b93-9650-b48f38270fbc" path="/var/lib/kubelet/pods/a2933953-2aba-4b93-9650-b48f38270fbc/volumes" Sep 19 09:58:58.510448 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:58.510414 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:58:58.611472 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:58:58.611433 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:02.711354 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:02.711315 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:02.812088 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:02.812046 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:02.911724 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:02.911688 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:02.965951 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:02.965864 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:03.109382 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:03.109347 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:03.209782 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:03.209743 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:05.114698 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:05.114671 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" Sep 19 09:59:08.966515 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:08.966479 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:09.027899 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:09.027858 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:17.562833 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:17.562792 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g"] Sep 19 09:59:17.563334 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:17.563011 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" podUID="f002f41d-3f45-4277-8be4-3b9875d78a24" containerName="manager" containerID="cri-o://65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c" gracePeriod=10 Sep 19 09:59:17.818159 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:17.818118 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" Sep 19 09:59:17.894425 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:17.894393 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f002f41d-3f45-4277-8be4-3b9875d78a24-extensions-socket-volume\") pod \"f002f41d-3f45-4277-8be4-3b9875d78a24\" (UID: \"f002f41d-3f45-4277-8be4-3b9875d78a24\") " Sep 19 09:59:17.894603 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:17.894446 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7btx\" (UniqueName: \"kubernetes.io/projected/f002f41d-3f45-4277-8be4-3b9875d78a24-kube-api-access-l7btx\") pod \"f002f41d-3f45-4277-8be4-3b9875d78a24\" (UID: \"f002f41d-3f45-4277-8be4-3b9875d78a24\") " Sep 19 09:59:17.894882 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:17.894857 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f002f41d-3f45-4277-8be4-3b9875d78a24-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f002f41d-3f45-4277-8be4-3b9875d78a24" (UID: "f002f41d-3f45-4277-8be4-3b9875d78a24"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 19 09:59:17.896718 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:17.896697 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f002f41d-3f45-4277-8be4-3b9875d78a24-kube-api-access-l7btx" (OuterVolumeSpecName: "kube-api-access-l7btx") pod "f002f41d-3f45-4277-8be4-3b9875d78a24" (UID: "f002f41d-3f45-4277-8be4-3b9875d78a24"). InnerVolumeSpecName "kube-api-access-l7btx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:59:17.995830 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:17.995789 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7btx\" (UniqueName: \"kubernetes.io/projected/f002f41d-3f45-4277-8be4-3b9875d78a24-kube-api-access-l7btx\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:59:17.995830 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:17.995820 2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f002f41d-3f45-4277-8be4-3b9875d78a24-extensions-socket-volume\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:59:18.220846 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:18.220814 2568 generic.go:356] "Generic (PLEG): container finished" podID="f002f41d-3f45-4277-8be4-3b9875d78a24" containerID="65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c" exitCode=0 Sep 19 09:59:18.221019 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:18.220856 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" Sep 19 09:59:18.221019 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:18.220864 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" event={"ID":"f002f41d-3f45-4277-8be4-3b9875d78a24","Type":"ContainerDied","Data":"65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c"} Sep 19 09:59:18.221019 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:18.220910 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g" event={"ID":"f002f41d-3f45-4277-8be4-3b9875d78a24","Type":"ContainerDied","Data":"fd8bbae17b4da52f4ffc760565992f5cbae073bf211ad4b3335e3e59db1cddc6"} Sep 19 09:59:18.221019 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:18.220932 2568 scope.go:117] "RemoveContainer" containerID="65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c" Sep 19 09:59:18.229409 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:18.229390 2568 scope.go:117] "RemoveContainer" containerID="65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c" Sep 19 09:59:18.229696 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:59:18.229675 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c\": container with ID starting with 65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c not found: ID does not exist" containerID="65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c" Sep 19 09:59:18.229765 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:18.229706 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c"} err="failed to get container status \"65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c\": rpc error: code = NotFound desc = could not find container \"65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c\": container with ID starting with 65b0613a52b0c8ac3fc7d49fb4d45071cfbb9c55356c87757f434110b7b6578c not found: ID does not exist" Sep 19 09:59:18.243593 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:18.243564 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g"] Sep 19 09:59:18.247552 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:18.247531 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-w5l7g"] Sep 19 09:59:20.085989 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:20.085957 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f002f41d-3f45-4277-8be4-3b9875d78a24" path="/var/lib/kubelet/pods/f002f41d-3f45-4277-8be4-3b9875d78a24/volumes" Sep 19 09:59:21.445573 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.445535 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp"] Sep 19 09:59:21.446225 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.446204 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f002f41d-3f45-4277-8be4-3b9875d78a24" containerName="manager" Sep 19 09:59:21.446225 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.446227 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f002f41d-3f45-4277-8be4-3b9875d78a24" containerName="manager" Sep 19 09:59:21.446392 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.446240 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2933953-2aba-4b93-9650-b48f38270fbc" containerName="manager" Sep 19 09:59:21.446392 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.446248 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2933953-2aba-4b93-9650-b48f38270fbc" containerName="manager" Sep 19 09:59:21.446392 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.446378 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2933953-2aba-4b93-9650-b48f38270fbc" containerName="manager" Sep 19 09:59:21.446540 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.446396 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f002f41d-3f45-4277-8be4-3b9875d78a24" containerName="manager" Sep 19 09:59:21.450504 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.450484 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.452912 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.452888 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-smb7g\"" Sep 19 09:59:21.453046 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.452892 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 19 09:59:21.460508 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.460480 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp"] Sep 19 09:59:21.470843 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.470810 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:21.526596 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.526562 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.526596 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.526597 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.526805 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.526628 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.526805 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.526688 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60eea589-007d-45ee-8bd2-679d3b510113-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.526805 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.526774 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.526915 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.526815 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.526915 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.526858 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60eea589-007d-45ee-8bd2-679d3b510113-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.526915 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.526889 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60eea589-007d-45ee-8bd2-679d3b510113-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.527006 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.526934 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sg29\" (UniqueName: \"kubernetes.io/projected/60eea589-007d-45ee-8bd2-679d3b510113-kube-api-access-4sg29\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.627941 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.627896 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.628178 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.627954 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.628178 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.627970 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60eea589-007d-45ee-8bd2-679d3b510113-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.628178 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.627996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.628178 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.628034 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.628178 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.628058 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60eea589-007d-45ee-8bd2-679d3b510113-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.628178 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.628083 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60eea589-007d-45ee-8bd2-679d3b510113-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.628178 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.628121 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sg29\" (UniqueName: \"kubernetes.io/projected/60eea589-007d-45ee-8bd2-679d3b510113-kube-api-access-4sg29\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.628555 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.628227 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.628555 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.628463 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.628555 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.628539 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.628698 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.628579 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.628898 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.628810 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/60eea589-007d-45ee-8bd2-679d3b510113-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.629037 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.629012 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.630652 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.630627 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/60eea589-007d-45ee-8bd2-679d3b510113-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.630842 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.630823 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/60eea589-007d-45ee-8bd2-679d3b510113-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.636234 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.636205 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/60eea589-007d-45ee-8bd2-679d3b510113-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.636532 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.636511 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sg29\" (UniqueName: \"kubernetes.io/projected/60eea589-007d-45ee-8bd2-679d3b510113-kube-api-access-4sg29\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zwvsp\" (UID: \"60eea589-007d-45ee-8bd2-679d3b510113\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:21.762314 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:21.762227 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:22.043207 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:22.043182 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp"] Sep 19 09:59:22.045607 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:59:22.045577 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60eea589_007d_45ee_8bd2_679d3b510113.slice/crio-20675c7fbfa0ddba39cf05397d8e4afae7341cc0446d0fec09c703b3a584a0e7 WatchSource:0}: Error finding container 20675c7fbfa0ddba39cf05397d8e4afae7341cc0446d0fec09c703b3a584a0e7: Status 404 returned error can't find the container with id 20675c7fbfa0ddba39cf05397d8e4afae7341cc0446d0fec09c703b3a584a0e7 Sep 19 09:59:22.082127 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:22.082095 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:22.248359 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:22.248328 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" event={"ID":"60eea589-007d-45ee-8bd2-679d3b510113","Type":"ContainerStarted","Data":"20675c7fbfa0ddba39cf05397d8e4afae7341cc0446d0fec09c703b3a584a0e7"} Sep 19 09:59:25.516058 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:25.516020 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"} Sep 19 09:59:25.516427 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:25.516096 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"} Sep 19 09:59:25.516427 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:25.516124 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"} Sep 19 09:59:26.271521 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:26.271482 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" event={"ID":"60eea589-007d-45ee-8bd2-679d3b510113","Type":"ContainerStarted","Data":"7f2a97075521b285a5bbc82e6c2bb5cd5ec64a4561962d957db00b03dbeceda1"} Sep 19 09:59:26.291375 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:26.291321 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" podStartSLOduration=1.82310235 podStartE2EDuration="5.291305662s" totalImagesPullingTime="3.468203312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:59:26.28912673 +0000 UTC m=+472.696638412" watchObservedRunningTime="2026-09-19 09:59:26.291305662 +0000 UTC m=+472.698817331" Sep 19 09:59:26.763052 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:26.763014 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:26.767976 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:26.767947 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:27.276844 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:27.276808 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:27.277740 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:27.277722 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 09:59:36.978482 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:36.978441 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:37.096481 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.096442 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:37.151441 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.151405 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:37.396770 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.396736 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-w5cqp"] Sep 19 09:59:37.401151 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.401119 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" Sep 19 09:59:37.403395 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.403374 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 19 09:59:37.403494 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.403452 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tpsml\"" Sep 19 09:59:37.403494 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.403450 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 19 09:59:37.403615 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.403450 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 19 09:59:37.410569 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.410547 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-w5cqp"] Sep 19 09:59:37.435520 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.435491 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-w5cqp"] Sep 19 09:59:37.471357 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.471332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f30d602a-7bf8-4897-b80f-b20c60aa78f9-config-file\") pod \"limitador-limitador-84994f7ffc-w5cqp\" (UID: \"f30d602a-7bf8-4897-b80f-b20c60aa78f9\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" Sep 19 09:59:37.471476 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.471371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m97p2\" (UniqueName: \"kubernetes.io/projected/f30d602a-7bf8-4897-b80f-b20c60aa78f9-kube-api-access-m97p2\") pod \"limitador-limitador-84994f7ffc-w5cqp\" (UID: \"f30d602a-7bf8-4897-b80f-b20c60aa78f9\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" Sep 19 09:59:37.572691 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.572652 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f30d602a-7bf8-4897-b80f-b20c60aa78f9-config-file\") pod \"limitador-limitador-84994f7ffc-w5cqp\" (UID: \"f30d602a-7bf8-4897-b80f-b20c60aa78f9\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" Sep 19 09:59:37.572691 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.572689 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m97p2\" (UniqueName: \"kubernetes.io/projected/f30d602a-7bf8-4897-b80f-b20c60aa78f9-kube-api-access-m97p2\") pod \"limitador-limitador-84994f7ffc-w5cqp\" (UID: \"f30d602a-7bf8-4897-b80f-b20c60aa78f9\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" Sep 19 09:59:37.573375 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.573347 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f30d602a-7bf8-4897-b80f-b20c60aa78f9-config-file\") pod \"limitador-limitador-84994f7ffc-w5cqp\" (UID: \"f30d602a-7bf8-4897-b80f-b20c60aa78f9\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" Sep 19 09:59:37.580439 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.580409 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m97p2\" (UniqueName: \"kubernetes.io/projected/f30d602a-7bf8-4897-b80f-b20c60aa78f9-kube-api-access-m97p2\") pod \"limitador-limitador-84994f7ffc-w5cqp\" (UID: \"f30d602a-7bf8-4897-b80f-b20c60aa78f9\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" Sep 19 09:59:37.711972 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.711887 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" Sep 19 09:59:37.791192 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.791162 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j8mpn"] Sep 19 09:59:37.794499 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.794477 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" Sep 19 09:59:37.796690 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.796669 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wmmgk\"" Sep 19 09:59:37.801237 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.801217 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j8mpn"] Sep 19 09:59:37.820399 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.820366 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:37.874594 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.874560 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc9zt\" (UniqueName: \"kubernetes.io/projected/2af3f492-23b0-4615-968f-99d978e96ffd-kube-api-access-hc9zt\") pod \"authorino-7bcfd8f9cd-j8mpn\" (UID: \"2af3f492-23b0-4615-968f-99d978e96ffd\") " pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" Sep 19 09:59:37.921512 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.921479 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:37.975371 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.975292 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc9zt\" (UniqueName: \"kubernetes.io/projected/2af3f492-23b0-4615-968f-99d978e96ffd-kube-api-access-hc9zt\") pod \"authorino-7bcfd8f9cd-j8mpn\" (UID: \"2af3f492-23b0-4615-968f-99d978e96ffd\") " pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" Sep 19 09:59:37.982215 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.982179 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-w5cqp"] Sep 19 09:59:37.984847 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:37.984817 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc9zt\" (UniqueName: \"kubernetes.io/projected/2af3f492-23b0-4615-968f-99d978e96ffd-kube-api-access-hc9zt\") pod \"authorino-7bcfd8f9cd-j8mpn\" (UID: \"2af3f492-23b0-4615-968f-99d978e96ffd\") " pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" Sep 19 09:59:37.985349 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:59:37.985324 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf30d602a_7bf8_4897_b80f_b20c60aa78f9.slice/crio-2fb782ad5cf3527ba4230fad4fac07d8bf591c75738f7788032053994d9acc96 WatchSource:0}: Error finding container 2fb782ad5cf3527ba4230fad4fac07d8bf591c75738f7788032053994d9acc96: Status 404 returned error can't find the container with id 2fb782ad5cf3527ba4230fad4fac07d8bf591c75738f7788032053994d9acc96 Sep 19 09:59:38.105425 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:38.105393 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" Sep 19 09:59:38.331122 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:38.331086 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" event={"ID":"f30d602a-7bf8-4897-b80f-b20c60aa78f9","Type":"ContainerStarted","Data":"2fb782ad5cf3527ba4230fad4fac07d8bf591c75738f7788032053994d9acc96"} Sep 19 09:59:38.376386 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:38.376362 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j8mpn"] Sep 19 09:59:38.378690 ip-10-0-139-28 kubenswrapper[2568]: W0919 09:59:38.378663 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2af3f492_23b0_4615_968f_99d978e96ffd.slice/crio-de16160ff2a61b6a272b3eae7d0da567afc15638618ef6338b5d588241d9e72f WatchSource:0}: Error finding container de16160ff2a61b6a272b3eae7d0da567afc15638618ef6338b5d588241d9e72f: Status 404 returned error can't find the container with id de16160ff2a61b6a272b3eae7d0da567afc15638618ef6338b5d588241d9e72f Sep 19 09:59:39.339205 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:39.339153 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" event={"ID":"2af3f492-23b0-4615-968f-99d978e96ffd","Type":"ContainerStarted","Data":"de16160ff2a61b6a272b3eae7d0da567afc15638618ef6338b5d588241d9e72f"} Sep 19 09:59:40.345708 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:40.345654 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" event={"ID":"2af3f492-23b0-4615-968f-99d978e96ffd","Type":"ContainerStarted","Data":"3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df"} Sep 19 09:59:40.347123 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:40.347097 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" event={"ID":"f30d602a-7bf8-4897-b80f-b20c60aa78f9","Type":"ContainerStarted","Data":"f572fa84cb769a6aa8d0490a6a543351a7edba2d2a97347c734f4fdd2d32b927"} Sep 19 09:59:40.347248 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:40.347179 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" Sep 19 09:59:40.360038 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:40.359991 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" podStartSLOduration=1.559426711 podStartE2EDuration="3.359977601s" totalImagesPullingTime="1.80055089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:59:40.359806815 +0000 UTC m=+486.767318485" watchObservedRunningTime="2026-09-19 09:59:40.359977601 +0000 UTC m=+486.767489273" Sep 19 09:59:40.378047 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:40.377988 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" podStartSLOduration=1.934884498 podStartE2EDuration="3.37797019s" totalImagesPullingTime="1.443085692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 09:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 09:59:40.37706881 +0000 UTC m=+486.784580501" watchObservedRunningTime="2026-09-19 09:59:40.37797019 +0000 UTC m=+486.785481859" Sep 19 09:59:40.717194 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:40.717161 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j8mpn"] Sep 19 09:59:42.357001 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:42.356938 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" podUID="2af3f492-23b0-4615-968f-99d978e96ffd" containerName="authorino" containerID="cri-o://3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df" gracePeriod=30 Sep 19 09:59:42.602733 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:42.602707 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" Sep 19 09:59:42.718467 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:42.718377 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc9zt\" (UniqueName: \"kubernetes.io/projected/2af3f492-23b0-4615-968f-99d978e96ffd-kube-api-access-hc9zt\") pod \"2af3f492-23b0-4615-968f-99d978e96ffd\" (UID: \"2af3f492-23b0-4615-968f-99d978e96ffd\") " Sep 19 09:59:42.720624 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:42.720597 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2af3f492-23b0-4615-968f-99d978e96ffd-kube-api-access-hc9zt" (OuterVolumeSpecName: "kube-api-access-hc9zt") pod "2af3f492-23b0-4615-968f-99d978e96ffd" (UID: "2af3f492-23b0-4615-968f-99d978e96ffd"). InnerVolumeSpecName "kube-api-access-hc9zt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 09:59:42.819465 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:42.819429 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc9zt\" (UniqueName: \"kubernetes.io/projected/2af3f492-23b0-4615-968f-99d978e96ffd-kube-api-access-hc9zt\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 09:59:43.362858 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:43.362823 2568 generic.go:356] "Generic (PLEG): container finished" podID="2af3f492-23b0-4615-968f-99d978e96ffd" containerID="3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df" exitCode=0 Sep 19 09:59:43.363270 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:43.362890 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" Sep 19 09:59:43.363270 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:43.362910 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" event={"ID":"2af3f492-23b0-4615-968f-99d978e96ffd","Type":"ContainerDied","Data":"3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df"} Sep 19 09:59:43.363270 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:43.362950 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-j8mpn" event={"ID":"2af3f492-23b0-4615-968f-99d978e96ffd","Type":"ContainerDied","Data":"de16160ff2a61b6a272b3eae7d0da567afc15638618ef6338b5d588241d9e72f"} Sep 19 09:59:43.363270 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:43.362966 2568 scope.go:117] "RemoveContainer" containerID="3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df" Sep 19 09:59:43.372021 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:43.372002 2568 scope.go:117] "RemoveContainer" containerID="3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df" Sep 19 09:59:43.372428 ip-10-0-139-28 kubenswrapper[2568]: E0919 09:59:43.372407 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df\": container with ID starting with 3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df not found: ID does not exist" containerID="3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df" Sep 19 09:59:43.372491 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:43.372440 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df"} err="failed to get container status \"3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df\": rpc error: code = NotFound desc = could not find container \"3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df\": container with ID starting with 3a16885ace14cd8a89c14cf178d7ad27531a2aa247dadaf127c583d7a25436df not found: ID does not exist" Sep 19 09:59:43.383342 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:43.383313 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j8mpn"] Sep 19 09:59:43.387440 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:43.387418 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-j8mpn"] Sep 19 09:59:44.086603 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:44.086575 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2af3f492-23b0-4615-968f-99d978e96ffd" path="/var/lib/kubelet/pods/2af3f492-23b0-4615-968f-99d978e96ffd/volumes" Sep 19 09:59:51.352838 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:51.352809 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-w5cqp" Sep 19 09:59:53.567493 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:53.567451 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:53.667959 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:53.667924 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 09:59:54.059113 ip-10-0-139-28 kubenswrapper[2568]: I0919 09:59:54.059013 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:07.116091 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.116052 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:07.341992 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.341961 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-cxv4c"] Sep 19 10:00:07.342344 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.342331 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2af3f492-23b0-4615-968f-99d978e96ffd" containerName="authorino" Sep 19 10:00:07.342396 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.342346 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2af3f492-23b0-4615-968f-99d978e96ffd" containerName="authorino" Sep 19 10:00:07.342437 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.342402 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2af3f492-23b0-4615-968f-99d978e96ffd" containerName="authorino" Sep 19 10:00:07.352072 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.352042 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-cxv4c"] Sep 19 10:00:07.352248 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.352168 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-cxv4c" Sep 19 10:00:07.354891 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.354860 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-wmmgk\"" Sep 19 10:00:07.410520 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.410429 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:07.420846 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.420820 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pgwz\" (UniqueName: \"kubernetes.io/projected/62b9a035-0a9c-4cdf-918c-0e6107db48c8-kube-api-access-9pgwz\") pod \"authorino-557f876f69-cxv4c\" (UID: \"62b9a035-0a9c-4cdf-918c-0e6107db48c8\") " pod="kuadrant-system/authorino-557f876f69-cxv4c" Sep 19 10:00:07.476580 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.476551 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-cxv4c"] Sep 19 10:00:07.476840 ip-10-0-139-28 kubenswrapper[2568]: E0919 10:00:07.476823 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-9pgwz], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-cxv4c" podUID="62b9a035-0a9c-4cdf-918c-0e6107db48c8" Sep 19 10:00:07.522205 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.522172 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pgwz\" (UniqueName: \"kubernetes.io/projected/62b9a035-0a9c-4cdf-918c-0e6107db48c8-kube-api-access-9pgwz\") pod \"authorino-557f876f69-cxv4c\" (UID: \"62b9a035-0a9c-4cdf-918c-0e6107db48c8\") " pod="kuadrant-system/authorino-557f876f69-cxv4c" Sep 19 10:00:07.531709 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:07.531677 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pgwz\" (UniqueName: \"kubernetes.io/projected/62b9a035-0a9c-4cdf-918c-0e6107db48c8-kube-api-access-9pgwz\") pod \"authorino-557f876f69-cxv4c\" (UID: \"62b9a035-0a9c-4cdf-918c-0e6107db48c8\") " pod="kuadrant-system/authorino-557f876f69-cxv4c" Sep 19 10:00:08.474463 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:08.474430 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-cxv4c" Sep 19 10:00:08.482049 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:08.482028 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-cxv4c" Sep 19 10:00:08.531124 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:08.531078 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pgwz\" (UniqueName: \"kubernetes.io/projected/62b9a035-0a9c-4cdf-918c-0e6107db48c8-kube-api-access-9pgwz\") pod \"62b9a035-0a9c-4cdf-918c-0e6107db48c8\" (UID: \"62b9a035-0a9c-4cdf-918c-0e6107db48c8\") " Sep 19 10:00:08.533259 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:08.533236 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62b9a035-0a9c-4cdf-918c-0e6107db48c8-kube-api-access-9pgwz" (OuterVolumeSpecName: "kube-api-access-9pgwz") pod "62b9a035-0a9c-4cdf-918c-0e6107db48c8" (UID: "62b9a035-0a9c-4cdf-918c-0e6107db48c8"). InnerVolumeSpecName "kube-api-access-9pgwz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 19 10:00:08.632410 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:08.632379 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pgwz\" (UniqueName: \"kubernetes.io/projected/62b9a035-0a9c-4cdf-918c-0e6107db48c8-kube-api-access-9pgwz\") on node \"ip-10-0-139-28.ec2.internal\" DevicePath \"\"" Sep 19 10:00:09.478323 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:09.478292 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-cxv4c" Sep 19 10:00:09.505522 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:09.505490 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-cxv4c"] Sep 19 10:00:09.509170 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:09.509109 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-cxv4c"] Sep 19 10:00:10.085459 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:10.085426 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62b9a035-0a9c-4cdf-918c-0e6107db48c8" path="/var/lib/kubelet/pods/62b9a035-0a9c-4cdf-918c-0e6107db48c8/volumes" Sep 19 10:00:15.515769 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:15.515731 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:15.580183 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:15.580151 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:15.633762 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:15.633727 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:15.718331 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:15.718294 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:15.772408 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:15.772315 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:18.991461 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:18.991420 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:19.091442 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:19.091404 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:25.466952 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:25.466851 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:25.527231 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:25.527185 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:50.638020 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:50.637976 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:50.731737 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:50.731704 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:52.200997 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:52.200959 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:52.301991 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:52.301960 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:52.362921 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:52.362871 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:52.515880 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:52.515760 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:52.762268 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:52.762232 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:52.861004 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:52.860968 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:00:52.969556 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:00:52.969522 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-w5cqp"] Sep 19 10:01:02.147819 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:02.147774 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:02.247356 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:02.247273 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:02.733467 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:02.733437 2568 generic.go:356] "Generic (PLEG): container finished" podID="60eea589-007d-45ee-8bd2-679d3b510113" containerID="7f2a97075521b285a5bbc82e6c2bb5cd5ec64a4561962d957db00b03dbeceda1" exitCode=0 Sep 19 10:01:02.733644 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:02.733511 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" event={"ID":"60eea589-007d-45ee-8bd2-679d3b510113","Type":"ContainerDied","Data":"7f2a97075521b285a5bbc82e6c2bb5cd5ec64a4561962d957db00b03dbeceda1"} Sep 19 10:01:02.733883 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:02.733870 2568 scope.go:117] "RemoveContainer" containerID="7f2a97075521b285a5bbc82e6c2bb5cd5ec64a4561962d957db00b03dbeceda1" Sep 19 10:01:02.734875 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:02.734843 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"} Sep 19 10:01:02.734996 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:02.734939 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"} Sep 19 10:01:02.734996 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:02.734988 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892140Ki","pods":"250"} Sep 19 10:01:03.744554 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:03.743642 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" event={"ID":"60eea589-007d-45ee-8bd2-679d3b510113","Type":"ContainerStarted","Data":"b0df1107e71e45d28e26ddb112b88a40ed9ce00fe10c171db099965c18e7a4f3"} Sep 19 10:01:03.768475 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:03.764644 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 10:01:03.768475 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:03.764780 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-zwvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Sep 19 10:01:03.768475 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:03.764850 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" podUID="60eea589-007d-45ee-8bd2-679d3b510113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Sep 19 10:01:04.763220 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:04.763183 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-zwvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Sep 19 10:01:04.763634 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:04.763247 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" podUID="60eea589-007d-45ee-8bd2-679d3b510113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Sep 19 10:01:05.762593 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:05.762564 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-zwvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Sep 19 10:01:05.762781 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:05.762617 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" podUID="60eea589-007d-45ee-8bd2-679d3b510113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Sep 19 10:01:06.762958 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:06.762920 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-zwvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Sep 19 10:01:06.763376 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:06.762986 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" podUID="60eea589-007d-45ee-8bd2-679d3b510113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Sep 19 10:01:07.762925 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:07.762874 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-zwvsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Sep 19 10:01:07.763106 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:07.762956 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" podUID="60eea589-007d-45ee-8bd2-679d3b510113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Sep 19 10:01:08.766304 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:08.766276 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 10:01:08.766778 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:08.766674 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 10:01:08.767422 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:08.767401 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zwvsp" Sep 19 10:01:09.495449 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:09.495419 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:09.864775 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:09.864734 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-w5cqp"] Sep 19 10:01:18.936434 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:18.936399 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj"] Sep 19 10:01:18.939536 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:18.939512 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:18.942269 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:18.942247 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 19 10:01:18.943098 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:18.943077 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sjfgs\"" Sep 19 10:01:18.943243 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:18.943077 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 19 10:01:18.943243 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:18.943114 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 19 10:01:18.950478 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:18.950450 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj"] Sep 19 10:01:18.975360 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:18.975324 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:19.071653 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.071615 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/648d08c6-544a-4049-91d5-fd76056073e4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.071850 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.071660 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/648d08c6-544a-4049-91d5-fd76056073e4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.071850 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.071699 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/648d08c6-544a-4049-91d5-fd76056073e4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.071850 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.071811 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkprf\" (UniqueName: \"kubernetes.io/projected/648d08c6-544a-4049-91d5-fd76056073e4-kube-api-access-vkprf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.071985 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.071862 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/648d08c6-544a-4049-91d5-fd76056073e4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.071985 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.071936 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/648d08c6-544a-4049-91d5-fd76056073e4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.173331 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.173294 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/648d08c6-544a-4049-91d5-fd76056073e4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.173511 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.173338 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/648d08c6-544a-4049-91d5-fd76056073e4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.173511 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.173357 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/648d08c6-544a-4049-91d5-fd76056073e4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.173511 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.173467 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/648d08c6-544a-4049-91d5-fd76056073e4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.173683 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.173551 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkprf\" (UniqueName: \"kubernetes.io/projected/648d08c6-544a-4049-91d5-fd76056073e4-kube-api-access-vkprf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.173683 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.173596 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/648d08c6-544a-4049-91d5-fd76056073e4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.173824 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.173802 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/648d08c6-544a-4049-91d5-fd76056073e4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.173889 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.173865 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/648d08c6-544a-4049-91d5-fd76056073e4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.173960 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.173928 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/648d08c6-544a-4049-91d5-fd76056073e4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.175829 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.175798 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/648d08c6-544a-4049-91d5-fd76056073e4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.176064 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.176049 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/648d08c6-544a-4049-91d5-fd76056073e4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.184562 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.184537 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkprf\" (UniqueName: \"kubernetes.io/projected/648d08c6-544a-4049-91d5-fd76056073e4-kube-api-access-vkprf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-hz9dj\" (UID: \"648d08c6-544a-4049-91d5-fd76056073e4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.253171 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.253053 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:19.523417 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.523394 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj"] Sep 19 10:01:19.525276 ip-10-0-139-28 kubenswrapper[2568]: W0919 10:01:19.525249 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648d08c6_544a_4049_91d5_fd76056073e4.slice/crio-0c120061c634eca5c16df6bfb3bf057f9a5da974ffd9e1d778f74f8332cde44c WatchSource:0}: Error finding container 0c120061c634eca5c16df6bfb3bf057f9a5da974ffd9e1d778f74f8332cde44c: Status 404 returned error can't find the container with id 0c120061c634eca5c16df6bfb3bf057f9a5da974ffd9e1d778f74f8332cde44c Sep 19 10:01:19.652868 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.652834 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-w5cqp"] Sep 19 10:01:19.830494 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:19.830456 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" event={"ID":"648d08c6-544a-4049-91d5-fd76056073e4","Type":"ContainerStarted","Data":"0c120061c634eca5c16df6bfb3bf057f9a5da974ffd9e1d778f74f8332cde44c"} Sep 19 10:01:20.837198 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:20.837158 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" event={"ID":"648d08c6-544a-4049-91d5-fd76056073e4","Type":"ContainerStarted","Data":"51f00ca81d5ad0b68590ffaacc9ef6d32e2d9fd699cfb36f4eb663f44aa3ece2"} Sep 19 10:01:20.856572 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:20.856518 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" podStartSLOduration=1.865664347 podStartE2EDuration="2.856500943s" totalImagesPullingTime="990.836596ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:01:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:01:20.855535117 +0000 UTC m=+587.263046790" watchObservedRunningTime="2026-09-19 10:01:20.856500943 +0000 UTC m=+587.264012609" Sep 19 10:01:24.669916 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:24.669885 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:25.760155 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:25.760105 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-w5cqp"] Sep 19 10:01:29.253807 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:29.253773 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:29.254225 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:29.253837 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:29.266289 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:29.266263 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:29.890201 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:29.890169 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-hz9dj" Sep 19 10:01:36.559011 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.558975 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs"] Sep 19 10:01:36.561981 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.561960 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.564594 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.564567 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 19 10:01:36.572103 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.572080 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs"] Sep 19 10:01:36.595498 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.595465 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:36.629708 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.629678 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e25918f-7471-4494-b550-e670c374d326-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.629856 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.629746 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e25918f-7471-4494-b550-e670c374d326-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.629856 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.629791 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nm86\" (UniqueName: \"kubernetes.io/projected/1e25918f-7471-4494-b550-e670c374d326-kube-api-access-5nm86\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.629955 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.629876 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e25918f-7471-4494-b550-e670c374d326-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.629955 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.629899 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e25918f-7471-4494-b550-e670c374d326-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.629955 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.629924 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e25918f-7471-4494-b550-e670c374d326-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.730875 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.730837 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e25918f-7471-4494-b550-e670c374d326-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.730875 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.730872 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nm86\" (UniqueName: \"kubernetes.io/projected/1e25918f-7471-4494-b550-e670c374d326-kube-api-access-5nm86\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.731089 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.731019 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e25918f-7471-4494-b550-e670c374d326-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.731089 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.731063 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e25918f-7471-4494-b550-e670c374d326-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.731236 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.731087 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e25918f-7471-4494-b550-e670c374d326-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.731236 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.731123 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e25918f-7471-4494-b550-e670c374d326-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.731436 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.731415 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e25918f-7471-4494-b550-e670c374d326-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.731511 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.731468 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e25918f-7471-4494-b550-e670c374d326-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.731595 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.731576 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e25918f-7471-4494-b550-e670c374d326-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.733412 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.733389 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e25918f-7471-4494-b550-e670c374d326-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.733546 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.733528 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e25918f-7471-4494-b550-e670c374d326-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.738917 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.738882 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nm86\" (UniqueName: \"kubernetes.io/projected/1e25918f-7471-4494-b550-e670c374d326-kube-api-access-5nm86\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlvfs\" (UID: \"1e25918f-7471-4494-b550-e670c374d326\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:36.874352 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:36.874319 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:37.145637 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:37.145537 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs"] Sep 19 10:01:37.148189 ip-10-0-139-28 kubenswrapper[2568]: W0919 10:01:37.148158 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e25918f_7471_4494_b550_e670c374d326.slice/crio-5a598eee532f35b961694102710001d9917ac75be4e6d6fe239eec36f37690bd WatchSource:0}: Error finding container 5a598eee532f35b961694102710001d9917ac75be4e6d6fe239eec36f37690bd: Status 404 returned error can't find the container with id 5a598eee532f35b961694102710001d9917ac75be4e6d6fe239eec36f37690bd Sep 19 10:01:37.254584 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:37.254551 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-w5cqp"] Sep 19 10:01:37.916893 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:37.916852 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" event={"ID":"1e25918f-7471-4494-b550-e670c374d326","Type":"ContainerStarted","Data":"fce4870d7ed97c211da206d7b3a0ea1b6e668314d204b82ab43bcc3dcb90085d"} Sep 19 10:01:37.917288 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:37.916920 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" event={"ID":"1e25918f-7471-4494-b550-e670c374d326","Type":"ContainerStarted","Data":"5a598eee532f35b961694102710001d9917ac75be4e6d6fe239eec36f37690bd"} Sep 19 10:01:37.936357 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:37.936307 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" podStartSLOduration=1.776713488 podStartE2EDuration="1.936287287s" totalImagesPullingTime="159.573799ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:01:37.935830844 +0000 UTC m=+604.343342557" watchObservedRunningTime="2026-09-19 10:01:37.936287287 +0000 UTC m=+604.343798956" Sep 19 10:01:42.475188 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.475147 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr"] Sep 19 10:01:42.477775 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.477751 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.480356 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.480334 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Sep 19 10:01:42.488165 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.488125 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr"] Sep 19 10:01:42.495633 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.495605 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:42.584936 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.584901 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd10b680-1abf-4ea7-a475-140a90915441-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.584936 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.584935 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd10b680-1abf-4ea7-a475-140a90915441-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.585223 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.584954 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd10b680-1abf-4ea7-a475-140a90915441-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.585223 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.585057 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm9vf\" (UniqueName: \"kubernetes.io/projected/dd10b680-1abf-4ea7-a475-140a90915441-kube-api-access-dm9vf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.585223 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.585102 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd10b680-1abf-4ea7-a475-140a90915441-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.585336 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.585236 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd10b680-1abf-4ea7-a475-140a90915441-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.686207 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.686175 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd10b680-1abf-4ea7-a475-140a90915441-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.686405 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.686247 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd10b680-1abf-4ea7-a475-140a90915441-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.686405 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.686265 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd10b680-1abf-4ea7-a475-140a90915441-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.686405 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.686283 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd10b680-1abf-4ea7-a475-140a90915441-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.686405 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.686327 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm9vf\" (UniqueName: \"kubernetes.io/projected/dd10b680-1abf-4ea7-a475-140a90915441-kube-api-access-dm9vf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.686405 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.686358 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd10b680-1abf-4ea7-a475-140a90915441-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.686676 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.686608 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd10b680-1abf-4ea7-a475-140a90915441-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.686676 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.686632 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd10b680-1abf-4ea7-a475-140a90915441-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.686827 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.686809 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd10b680-1abf-4ea7-a475-140a90915441-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.688802 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.688781 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd10b680-1abf-4ea7-a475-140a90915441-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.688950 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.688931 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd10b680-1abf-4ea7-a475-140a90915441-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.707828 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.707798 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm9vf\" (UniqueName: \"kubernetes.io/projected/dd10b680-1abf-4ea7-a475-140a90915441-kube-api-access-dm9vf\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr\" (UID: \"dd10b680-1abf-4ea7-a475-140a90915441\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:42.789474 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:42.789384 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:43.064434 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:43.064408 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr"] Sep 19 10:01:43.067114 ip-10-0-139-28 kubenswrapper[2568]: W0919 10:01:43.067087 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd10b680_1abf_4ea7_a475_140a90915441.slice/crio-c8563ec36dc45994e3b625abfe6e1519f010cd084ee196ca9983a4bde8caaae4 WatchSource:0}: Error finding container c8563ec36dc45994e3b625abfe6e1519f010cd084ee196ca9983a4bde8caaae4: Status 404 returned error can't find the container with id c8563ec36dc45994e3b625abfe6e1519f010cd084ee196ca9983a4bde8caaae4 Sep 19 10:01:43.373369 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:43.373284 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:43.473808 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:43.473766 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:43.726437 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:43.726350 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:43.780968 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:43.780919 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:43.949751 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:43.949717 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" event={"ID":"dd10b680-1abf-4ea7-a475-140a90915441","Type":"ContainerStarted","Data":"cd049e1c7b4d3bf58b13409b3b4484825382f584d4a414f4fc232c8b7f7db6d4"} Sep 19 10:01:43.949950 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:43.949778 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" event={"ID":"dd10b680-1abf-4ea7-a475-140a90915441","Type":"ContainerStarted","Data":"c8563ec36dc45994e3b625abfe6e1519f010cd084ee196ca9983a4bde8caaae4"} Sep 19 10:01:43.969633 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:43.969576 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" podStartSLOduration=1.81300252 podStartE2EDuration="1.969558397s" totalImagesPullingTime="156.555877ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:01:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:01:43.968360538 +0000 UTC m=+610.375872243" watchObservedRunningTime="2026-09-19 10:01:43.969558397 +0000 UTC m=+610.377070066" Sep 19 10:01:44.157803 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:44.157770 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-w5cqp"] Sep 19 10:01:46.874445 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:46.874399 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:46.874874 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:46.874461 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:46.886850 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:46.886829 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:46.976655 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:46.976629 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlvfs" Sep 19 10:01:51.750435 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.750403 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj"] Sep 19 10:01:51.755181 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.755159 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.758619 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.758597 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Sep 19 10:01:51.776049 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.776025 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj"] Sep 19 10:01:51.796496 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.796462 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:51.872275 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.872241 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6c78064-8159-4e3e-9714-ad34263a46ca-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.872441 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.872279 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2dhk\" (UniqueName: \"kubernetes.io/projected/d6c78064-8159-4e3e-9714-ad34263a46ca-kube-api-access-s2dhk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.872441 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.872360 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6c78064-8159-4e3e-9714-ad34263a46ca-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.872441 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.872415 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6c78064-8159-4e3e-9714-ad34263a46ca-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.872551 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.872465 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6c78064-8159-4e3e-9714-ad34263a46ca-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.872551 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.872486 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6c78064-8159-4e3e-9714-ad34263a46ca-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.973831 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.973792 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6c78064-8159-4e3e-9714-ad34263a46ca-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.974021 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.973847 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6c78064-8159-4e3e-9714-ad34263a46ca-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.974021 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.973893 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6c78064-8159-4e3e-9714-ad34263a46ca-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.974021 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.973911 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6c78064-8159-4e3e-9714-ad34263a46ca-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.974021 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.973937 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6c78064-8159-4e3e-9714-ad34263a46ca-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.974021 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.973962 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2dhk\" (UniqueName: \"kubernetes.io/projected/d6c78064-8159-4e3e-9714-ad34263a46ca-kube-api-access-s2dhk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.974328 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.974303 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6c78064-8159-4e3e-9714-ad34263a46ca-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.974390 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.974309 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6c78064-8159-4e3e-9714-ad34263a46ca-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.974477 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.974456 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6c78064-8159-4e3e-9714-ad34263a46ca-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.976479 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.976463 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6c78064-8159-4e3e-9714-ad34263a46ca-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.976552 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.976535 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6c78064-8159-4e3e-9714-ad34263a46ca-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:51.983726 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:51.983707 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2dhk\" (UniqueName: \"kubernetes.io/projected/d6c78064-8159-4e3e-9714-ad34263a46ca-kube-api-access-s2dhk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6dftj\" (UID: \"d6c78064-8159-4e3e-9714-ad34263a46ca\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:52.066920 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:52.066872 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:01:52.339661 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:52.339637 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj"] Sep 19 10:01:52.342306 ip-10-0-139-28 kubenswrapper[2568]: W0919 10:01:52.342272 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6c78064_8159_4e3e_9714_ad34263a46ca.slice/crio-1cfb4004ebc020b24c27ff1fe7056fa57e82d73ee2ae2ccffa99a54b9fae41bb WatchSource:0}: Error finding container 1cfb4004ebc020b24c27ff1fe7056fa57e82d73ee2ae2ccffa99a54b9fae41bb: Status 404 returned error can't find the container with id 1cfb4004ebc020b24c27ff1fe7056fa57e82d73ee2ae2ccffa99a54b9fae41bb Sep 19 10:01:52.342735 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:52.342691 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:01:52.343921 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:52.343904 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 19 10:01:52.790507 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:52.790415 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:52.790507 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:52.790489 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:52.803085 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:52.803058 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:52.998027 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:52.997994 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" event={"ID":"d6c78064-8159-4e3e-9714-ad34263a46ca","Type":"ContainerStarted","Data":"0832227758aa5930a526335dc81675e8fcd6c3f54c3a640b98e9aff8b66ed008"} Sep 19 10:01:52.998240 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:52.998055 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" event={"ID":"d6c78064-8159-4e3e-9714-ad34263a46ca","Type":"ContainerStarted","Data":"1cfb4004ebc020b24c27ff1fe7056fa57e82d73ee2ae2ccffa99a54b9fae41bb"} Sep 19 10:01:53.009296 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:53.009272 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr" Sep 19 10:01:53.017756 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:53.017715 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" podStartSLOduration=1.832820933 podStartE2EDuration="2.017700759s" totalImagesPullingTime="184.879826ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:01:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:01:53.017336949 +0000 UTC m=+619.424848617" watchObservedRunningTime="2026-09-19 10:01:53.017700759 +0000 UTC m=+619.425212427" Sep 19 10:01:53.564456 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:01:53.564416 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-w5cqp"] Sep 19 10:02:01.152285 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.152239 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k"] Sep 19 10:02:01.155910 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.155895 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.158590 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.158564 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Sep 19 10:02:01.165622 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.165595 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k"] Sep 19 10:02:01.194380 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.194346 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:02:01.258440 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.258405 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc21cff-4713-4501-a793-8254c7980b7e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.258622 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.258447 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc21cff-4713-4501-a793-8254c7980b7e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.258622 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.258510 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc21cff-4713-4501-a793-8254c7980b7e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.258622 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.258542 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9llb\" (UniqueName: \"kubernetes.io/projected/5fc21cff-4713-4501-a793-8254c7980b7e-kube-api-access-l9llb\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.258622 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.258577 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc21cff-4713-4501-a793-8254c7980b7e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.258622 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.258602 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc21cff-4713-4501-a793-8254c7980b7e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.360023 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.359987 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc21cff-4713-4501-a793-8254c7980b7e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.360023 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.360028 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc21cff-4713-4501-a793-8254c7980b7e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.360302 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.360059 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc21cff-4713-4501-a793-8254c7980b7e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.360302 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.360194 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9llb\" (UniqueName: \"kubernetes.io/projected/5fc21cff-4713-4501-a793-8254c7980b7e-kube-api-access-l9llb\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.360302 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.360259 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc21cff-4713-4501-a793-8254c7980b7e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.360302 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.360293 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc21cff-4713-4501-a793-8254c7980b7e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.360522 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.360462 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fc21cff-4713-4501-a793-8254c7980b7e-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.360578 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.360523 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fc21cff-4713-4501-a793-8254c7980b7e-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.360619 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.360602 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fc21cff-4713-4501-a793-8254c7980b7e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.362995 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.362968 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fc21cff-4713-4501-a793-8254c7980b7e-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.363163 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.363001 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fc21cff-4713-4501-a793-8254c7980b7e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.369105 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.369082 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9llb\" (UniqueName: \"kubernetes.io/projected/5fc21cff-4713-4501-a793-8254c7980b7e-kube-api-access-l9llb\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k\" (UID: \"5fc21cff-4713-4501-a793-8254c7980b7e\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.467453 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.467357 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:01.740228 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:01.740203 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k"] Sep 19 10:02:01.742016 ip-10-0-139-28 kubenswrapper[2568]: W0919 10:02:01.741992 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fc21cff_4713_4501_a793_8254c7980b7e.slice/crio-0a28230a59d1301329748ebc55dfe3731c131c858e10404f7033cc0c28acd862 WatchSource:0}: Error finding container 0a28230a59d1301329748ebc55dfe3731c131c858e10404f7033cc0c28acd862: Status 404 returned error can't find the container with id 0a28230a59d1301329748ebc55dfe3731c131c858e10404f7033cc0c28acd862 Sep 19 10:02:02.042855 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:02.042767 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" event={"ID":"5fc21cff-4713-4501-a793-8254c7980b7e","Type":"ContainerStarted","Data":"6ba750bfdf4dde79356c38e81b50577e1d2716ec302a1ac3022aca28bb8c8906"} Sep 19 10:02:02.042855 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:02.042815 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" event={"ID":"5fc21cff-4713-4501-a793-8254c7980b7e","Type":"ContainerStarted","Data":"0a28230a59d1301329748ebc55dfe3731c131c858e10404f7033cc0c28acd862"} Sep 19 10:02:02.062117 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:02.062061 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" podStartSLOduration=0.892074563 podStartE2EDuration="1.062042798s" totalImagesPullingTime="169.968235ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 10:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:02:02.05980703 +0000 UTC m=+628.467318718" watchObservedRunningTime="2026-09-19 10:02:02.062042798 +0000 UTC m=+628.469554469" Sep 19 10:02:02.067110 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:02.067085 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:02:02.067241 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:02.067173 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:02:02.093515 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:02.090764 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:02:03.058198 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:03.058171 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6dftj" Sep 19 10:02:03.350032 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:03.349998 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-w5cqp"] Sep 19 10:02:11.468496 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:11.468459 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:11.468949 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:11.468544 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:11.481225 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:11.481199 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:12.100063 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:12.100032 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k" Sep 19 10:02:33.886871 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:33.886841 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:02:33.986814 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:02:33.986773 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:03:20.488028 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:20.487997 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:03:20.554219 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:20.554176 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:03:21.352245 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:21.352205 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:03:21.443940 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:21.443904 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:03:27.737888 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:27.737812 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm_e979c06f-682d-4d31-a021-85f46c825383/copy-manifests/0.log" Sep 19 10:03:27.746985 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:27.746961 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm_e979c06f-682d-4d31-a021-85f46c825383/manager/0.log" Sep 19 10:03:29.777833 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:29.777805 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-w5cqp_f30d602a-7bf8-4897-b80f-b20c60aa78f9/limitador/0.log" Sep 19 10:03:30.474358 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:30.474324 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65d7d84b5d-4j8rz_3e40e2e2-b805-47b0-88e5-9fcbdd3a843c/kube-auth-proxy/0.log" Sep 19 10:03:30.724085 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:30.724043 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zwvsp_60eea589-007d-45ee-8bd2-679d3b510113/istio-proxy/1.log" Sep 19 10:03:31.505540 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:31.505507 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-hz9dj_648d08c6-544a-4049-91d5-fd76056073e4/main/0.log" Sep 19 10:03:31.744034 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:31.743996 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-wlvfs_1e25918f-7471-4494-b550-e670c374d326/main/0.log" Sep 19 10:03:31.866347 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:31.866316 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr_dd10b680-1abf-4ea7-a475-140a90915441/main/0.log" Sep 19 10:03:31.987023 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:31.986992 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6dftj_d6c78064-8159-4e3e-9714-ad34263a46ca/main/0.log" Sep 19 10:03:32.104915 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:32.104889 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k_5fc21cff-4713-4501-a793-8254c7980b7e/main/0.log" Sep 19 10:03:39.279785 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:39.279754 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm_e979c06f-682d-4d31-a021-85f46c825383/copy-manifests/0.log" Sep 19 10:03:39.288824 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:39.288794 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm_e979c06f-682d-4d31-a021-85f46c825383/manager/0.log" Sep 19 10:03:41.227989 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:41.227957 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-w5cqp_f30d602a-7bf8-4897-b80f-b20c60aa78f9/limitador/0.log" Sep 19 10:03:41.918832 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:41.918804 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65d7d84b5d-4j8rz_3e40e2e2-b805-47b0-88e5-9fcbdd3a843c/kube-auth-proxy/0.log" Sep 19 10:03:42.143340 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:42.143309 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zwvsp_60eea589-007d-45ee-8bd2-679d3b510113/istio-proxy/1.log" Sep 19 10:03:42.974531 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:42.974499 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-hz9dj_648d08c6-544a-4049-91d5-fd76056073e4/main/0.log" Sep 19 10:03:43.200253 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:43.200219 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-wlvfs_1e25918f-7471-4494-b550-e670c374d326/main/0.log" Sep 19 10:03:43.304747 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:43.304669 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fbnwmr_dd10b680-1abf-4ea7-a475-140a90915441/main/0.log" Sep 19 10:03:43.415937 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:43.415907 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6dftj_d6c78064-8159-4e3e-9714-ad34263a46ca/main/0.log" Sep 19 10:03:43.527824 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:43.527795 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-bzl9k_5fc21cff-4713-4501-a793-8254c7980b7e/main/0.log" Sep 19 10:03:59.908377 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:03:59.908342 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:04:00.003329 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:00.003296 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:04:03.015265 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:03.015239 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pzsvj_3875e600-bb82-4e1a-8e93-3afae30494f7/global-pull-secret-syncer/0.log" Sep 19 10:04:03.073158 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:03.073108 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" Sep 19 10:04:03.173956 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:03.173919 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-28.ec2.internal_fcbba51457286c61d2f15a5467b9e8e5/haproxy/0.log" Sep 19 10:04:07.684717 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:07.684684 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-w5cqp_f30d602a-7bf8-4897-b80f-b20c60aa78f9/limitador/0.log" Sep 19 10:04:09.574039 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:09.574010 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-z556b_35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c/kube-state-metrics/0.log" Sep 19 10:04:09.605157 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:09.605109 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-z556b_35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c/kube-rbac-proxy-main/0.log" Sep 19 10:04:09.641471 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:09.641434 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-z556b_35ac82b9-a6b6-4e8c-8c95-eaf19d23ef0c/kube-rbac-proxy-self/0.log" Sep 19 10:04:09.820197 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:09.820169 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nv4ww_f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe/node-exporter/0.log" Sep 19 10:04:09.844281 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:09.844209 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nv4ww_f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe/kube-rbac-proxy/0.log" Sep 19 10:04:09.877850 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:09.877823 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-nv4ww_f2e69fd6-e028-4fb6-b92b-9aefc0ee9cbe/init-textfile/0.log" Sep 19 10:04:10.020538 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.020514 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-v7dq5_f72d1c86-a71e-4109-af38-e690580b09e3/kube-rbac-proxy-main/0.log" Sep 19 10:04:10.061221 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.061187 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-v7dq5_f72d1c86-a71e-4109-af38-e690580b09e3/kube-rbac-proxy-self/0.log" Sep 19 10:04:10.097615 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.097517 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-v7dq5_f72d1c86-a71e-4109-af38-e690580b09e3/openshift-state-metrics/0.log" Sep 19 10:04:10.146728 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.146702 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8905a6a4-dc65-429d-a392-7c4f396f8acd/prometheus/0.log" Sep 19 10:04:10.173654 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.173627 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8905a6a4-dc65-429d-a392-7c4f396f8acd/config-reloader/0.log" Sep 19 10:04:10.196338 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.196311 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8905a6a4-dc65-429d-a392-7c4f396f8acd/thanos-sidecar/0.log" Sep 19 10:04:10.220057 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.220031 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8905a6a4-dc65-429d-a392-7c4f396f8acd/kube-rbac-proxy-web/0.log" Sep 19 10:04:10.242335 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.242305 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8905a6a4-dc65-429d-a392-7c4f396f8acd/kube-rbac-proxy/0.log" Sep 19 10:04:10.267938 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.267913 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8905a6a4-dc65-429d-a392-7c4f396f8acd/kube-rbac-proxy-thanos/0.log" Sep 19 10:04:10.290305 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.290278 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8905a6a4-dc65-429d-a392-7c4f396f8acd/init-config-reloader/0.log" Sep 19 10:04:10.331757 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.331731 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-tgn9p_83d661af-a331-4da5-9998-6afba67a012f/prometheus-operator/0.log" Sep 19 10:04:10.371476 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.371391 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-tgn9p_83d661af-a331-4da5-9998-6afba67a012f/kube-rbac-proxy/0.log" Sep 19 10:04:10.398636 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.398605 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5c6747dc7b-wqqg5_9b05182f-9d1d-40e1-8c00-206a0c647071/prometheus-operator-admission-webhook/0.log" Sep 19 10:04:10.429479 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.429446 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8567b5c8df-g9fl7_925d61a7-0c16-4b05-9e85-6cb8c0c0518f/telemeter-client/0.log" Sep 19 10:04:10.457076 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.457036 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8567b5c8df-g9fl7_925d61a7-0c16-4b05-9e85-6cb8c0c0518f/reload/0.log" Sep 19 10:04:10.489803 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.489775 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8567b5c8df-g9fl7_925d61a7-0c16-4b05-9e85-6cb8c0c0518f/kube-rbac-proxy/0.log" Sep 19 10:04:10.531639 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.531612 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9ffdd76d9-dmbck_5e56f2cd-f694-4c15-ae09-4ba427845ad9/thanos-query/0.log" Sep 19 10:04:10.559288 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.559265 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9ffdd76d9-dmbck_5e56f2cd-f694-4c15-ae09-4ba427845ad9/kube-rbac-proxy-web/0.log" Sep 19 10:04:10.586330 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.586308 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9ffdd76d9-dmbck_5e56f2cd-f694-4c15-ae09-4ba427845ad9/kube-rbac-proxy/0.log" Sep 19 10:04:10.608739 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.608718 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9ffdd76d9-dmbck_5e56f2cd-f694-4c15-ae09-4ba427845ad9/prom-label-proxy/0.log" Sep 19 10:04:10.632004 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.631925 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9ffdd76d9-dmbck_5e56f2cd-f694-4c15-ae09-4ba427845ad9/kube-rbac-proxy-rules/0.log" Sep 19 10:04:10.658814 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:10.658783 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-9ffdd76d9-dmbck_5e56f2cd-f694-4c15-ae09-4ba427845ad9/kube-rbac-proxy-metrics/0.log" Sep 19 10:04:12.581780 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.581749 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw"] Sep 19 10:04:12.585617 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.585598 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.588051 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.588034 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2l7hw\"/\"kube-root-ca.crt\"" Sep 19 10:04:12.588180 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.588073 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2l7hw\"/\"openshift-service-ca.crt\"" Sep 19 10:04:12.588817 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.588804 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2l7hw\"/\"default-dockercfg-5zqvl\"" Sep 19 10:04:12.595270 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.595247 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw"] Sep 19 10:04:12.609904 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.609869 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b83d9639-89c2-4091-86dc-908826f9fd46-sys\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.610060 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.610044 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b83d9639-89c2-4091-86dc-908826f9fd46-podres\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.610206 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.610174 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b83d9639-89c2-4091-86dc-908826f9fd46-proc\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.610298 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.610253 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b83d9639-89c2-4091-86dc-908826f9fd46-lib-modules\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.610456 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.610414 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfpbw\" (UniqueName: \"kubernetes.io/projected/b83d9639-89c2-4091-86dc-908826f9fd46-kube-api-access-xfpbw\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.711628 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.711587 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b83d9639-89c2-4091-86dc-908826f9fd46-lib-modules\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.711823 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.711711 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfpbw\" (UniqueName: \"kubernetes.io/projected/b83d9639-89c2-4091-86dc-908826f9fd46-kube-api-access-xfpbw\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.711823 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.711744 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b83d9639-89c2-4091-86dc-908826f9fd46-sys\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.711823 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.711750 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b83d9639-89c2-4091-86dc-908826f9fd46-lib-modules\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.711823 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.711777 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b83d9639-89c2-4091-86dc-908826f9fd46-podres\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.711823 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.711802 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b83d9639-89c2-4091-86dc-908826f9fd46-sys\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.711823 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.711802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b83d9639-89c2-4091-86dc-908826f9fd46-proc\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.712060 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.711853 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b83d9639-89c2-4091-86dc-908826f9fd46-proc\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.712060 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.711878 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b83d9639-89c2-4091-86dc-908826f9fd46-podres\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.720435 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.720418 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfpbw\" (UniqueName: \"kubernetes.io/projected/b83d9639-89c2-4091-86dc-908826f9fd46-kube-api-access-xfpbw\") pod \"perf-node-gather-daemonset-mmkkw\" (UID: \"b83d9639-89c2-4091-86dc-908826f9fd46\") " pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:12.896716 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:12.896622 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:13.175049 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:13.175024 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw"] Sep 19 10:04:13.177740 ip-10-0-139-28 kubenswrapper[2568]: W0919 10:04:13.177707 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb83d9639_89c2_4091_86dc_908826f9fd46.slice/crio-cf3946da8cc66e37fff9e892588189d0263246f24216b7abf471ad20a0be6ea9 WatchSource:0}: Error finding container cf3946da8cc66e37fff9e892588189d0263246f24216b7abf471ad20a0be6ea9: Status 404 returned error can't find the container with id cf3946da8cc66e37fff9e892588189d0263246f24216b7abf471ad20a0be6ea9 Sep 19 10:04:13.373122 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:13.373093 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5d6556bd68-pt9sg_e08ca42f-5053-4609-b195-53bb7ce1925b/console/0.log" Sep 19 10:04:13.449630 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:13.449546 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6594f4c75-kq2bf_55bd529d-d87f-4f3c-9373-c11da5bacf17/download-server/0.log" Sep 19 10:04:13.675759 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:13.675729 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" event={"ID":"b83d9639-89c2-4091-86dc-908826f9fd46","Type":"ContainerStarted","Data":"0ea7eea75ff9cccb3690b01321e57d26c1747fc71dc7aa1b01eaf1e99cdc388c"} Sep 19 10:04:13.676276 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:13.675779 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" event={"ID":"b83d9639-89c2-4091-86dc-908826f9fd46","Type":"ContainerStarted","Data":"cf3946da8cc66e37fff9e892588189d0263246f24216b7abf471ad20a0be6ea9"} Sep 19 10:04:13.676276 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:13.675845 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:13.700876 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:13.700782 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" podStartSLOduration=1.7007644800000001 podStartE2EDuration="1.70076448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 10:04:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 10:04:13.700754717 +0000 UTC m=+760.108266384" watchObservedRunningTime="2026-09-19 10:04:13.70076448 +0000 UTC m=+760.108276183" Sep 19 10:04:15.194887 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:15.194851 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c9z9x_558cb6d2-cd50-4660-938f-b516d719e585/dns/0.log" Sep 19 10:04:15.218248 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:15.218221 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c9z9x_558cb6d2-cd50-4660-938f-b516d719e585/kube-rbac-proxy/0.log" Sep 19 10:04:15.371777 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:15.371743 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jddfm_b3d2ddbc-feb3-4edb-9713-5d1822cce2b8/dns-node-resolver/0.log" Sep 19 10:04:15.943846 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:15.943817 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-24d99_75b47a48-c5ec-4502-b447-6c6dccf8b622/node-ca/0.log" Sep 19 10:04:17.063230 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:17.063203 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65d7d84b5d-4j8rz_3e40e2e2-b805-47b0-88e5-9fcbdd3a843c/kube-auth-proxy/0.log" Sep 19 10:04:17.117447 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:17.117414 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zwvsp_60eea589-007d-45ee-8bd2-679d3b510113/istio-proxy/1.log" Sep 19 10:04:17.118201 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:17.118179 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zwvsp_60eea589-007d-45ee-8bd2-679d3b510113/istio-proxy/0.log" Sep 19 10:04:17.266517 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:17.266485 2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-79pp9_4dad002c-07a5-481d-8f21-b3f4ad57a2c0/konnectivity-agent/0.log" limit=1024 Sep 19 10:04:17.829466 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:17.829434 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2bz4b_b3a8074b-9370-478d-831e-ed4b4b648f35/serve-healthcheck-canary/0.log" Sep 19 10:04:18.460312 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:18.460283 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g8sm5_c9cbd823-b438-485f-b798-60658d28fd38/kube-rbac-proxy/0.log" Sep 19 10:04:18.481066 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:18.481037 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g8sm5_c9cbd823-b438-485f-b798-60658d28fd38/exporter/0.log" Sep 19 10:04:18.503212 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:18.503186 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g8sm5_c9cbd823-b438-485f-b798-60658d28fd38/extractor/0.log" Sep 19 10:04:19.690373 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:19.690349 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2l7hw/perf-node-gather-daemonset-mmkkw" Sep 19 10:04:20.975752 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:20.975717 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm_e979c06f-682d-4d31-a021-85f46c825383/manager/0.log" Sep 19 10:04:20.995526 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:20.995502 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c8c4cc7bf-x7drm_e979c06f-682d-4d31-a021-85f46c825383/copy-manifests/0.log" Sep 19 10:04:22.572831 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:22.572803 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84d7567998-v2cts_2951106a-dc2f-4d40-add3-7bd5ec37906d/manager/0.log" Sep 19 10:04:23.424740 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:23.424703 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65d7d84b5d-4j8rz_3e40e2e2-b805-47b0-88e5-9fcbdd3a843c/kube-auth-proxy/0.log" Sep 19 10:04:23.466192 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:23.466156 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zwvsp_60eea589-007d-45ee-8bd2-679d3b510113/istio-proxy/1.log" Sep 19 10:04:23.466971 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:23.466947 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zwvsp_60eea589-007d-45ee-8bd2-679d3b510113/istio-proxy/0.log" Sep 19 10:04:24.692671 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:24.692586 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65d7d84b5d-4j8rz_3e40e2e2-b805-47b0-88e5-9fcbdd3a843c/kube-auth-proxy/0.log" Sep 19 10:04:24.731996 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:24.731955 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zwvsp_60eea589-007d-45ee-8bd2-679d3b510113/istio-proxy/1.log" Sep 19 10:04:24.732497 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:24.732478 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zwvsp_60eea589-007d-45ee-8bd2-679d3b510113/istio-proxy/0.log" Sep 19 10:04:29.399696 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:29.399666 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bxkqq_8f492656-f7cf-4308-ab65-fe7b5f59c31b/kube-multus-additional-cni-plugins/0.log" Sep 19 10:04:29.422852 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:29.422810 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bxkqq_8f492656-f7cf-4308-ab65-fe7b5f59c31b/egress-router-binary-copy/0.log" Sep 19 10:04:29.454245 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:29.454206 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bxkqq_8f492656-f7cf-4308-ab65-fe7b5f59c31b/cni-plugins/0.log" Sep 19 10:04:29.480674 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:29.480638 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bxkqq_8f492656-f7cf-4308-ab65-fe7b5f59c31b/bond-cni-plugin/0.log" Sep 19 10:04:29.503648 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:29.503619 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bxkqq_8f492656-f7cf-4308-ab65-fe7b5f59c31b/routeoverride-cni/0.log" Sep 19 10:04:29.527082 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:29.527054 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bxkqq_8f492656-f7cf-4308-ab65-fe7b5f59c31b/whereabouts-cni-bincopy/0.log" Sep 19 10:04:29.549383 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:29.549355 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bxkqq_8f492656-f7cf-4308-ab65-fe7b5f59c31b/whereabouts-cni/0.log" Sep 19 10:04:29.762543 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:29.762469 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-pllt7_6bd97875-f073-455a-bbaa-17fa3c598cbe/kube-multus/0.log" Sep 19 10:04:29.874100 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:29.874071 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hrrnt_0559c450-7285-4470-9fb8-12716eff996a/network-metrics-daemon/0.log" Sep 19 10:04:29.893632 ip-10-0-139-28 kubenswrapper[2568]: I0919 10:04:29.893593 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-hrrnt_0559c450-7285-4470-9fb8-12716eff996a/kube-rbac-proxy/0.log"