Sep 09 21:08:08.341993 ip-10-0-137-115 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Sep 09 21:08:08.342005 ip-10-0-137-115 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Sep 09 21:08:08.342012 ip-10-0-137-115 systemd[1]: kubelet.service: Failed with result 'resources'. Sep 09 21:08:08.342148 ip-10-0-137-115 systemd[1]: Failed to start Kubernetes Kubelet. Sep 09 21:08:19.645930 ip-10-0-137-115 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Sep 09 21:08:19.645944 ip-10-0-137-115 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot d9b3d1625dc74902b40d973ca983b661 -- Sep 09 21:09:49.766386 ip-10-0-137-115 systemd[1]: Starting Kubernetes Kubelet... Sep 09 21:09:50.212728 ip-10-0-137-115 kubenswrapper[2553]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 21:09:50.212728 ip-10-0-137-115 kubenswrapper[2553]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Sep 09 21:09:50.212728 ip-10-0-137-115 kubenswrapper[2553]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 21:09:50.212728 ip-10-0-137-115 kubenswrapper[2553]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Sep 09 21:09:50.212728 ip-10-0-137-115 kubenswrapper[2553]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Sep 09 21:09:50.214755 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.214668 2553 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Sep 09 21:09:50.218877 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218860 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:09:50.218877 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218877 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218882 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218885 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218888 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218891 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218894 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218897 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218900 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218902 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218905 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218908 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218911 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218914 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218916 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218919 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218921 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218930 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218933 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218936 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218939 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218942 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:09:50.218942 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218945 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218948 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218950 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218953 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218957 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218960 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218962 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218966 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218968 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218971 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218973 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218976 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218979 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218982 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218986 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218990 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218992 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218995 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.218999 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:09:50.219455 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219002 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219006 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219008 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219011 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219014 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219016 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219019 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219021 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219024 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219027 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219030 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219032 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219762 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219768 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219771 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219774 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219777 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219780 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219783 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219786 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:09:50.219945 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219789 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219792 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219794 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219797 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219799 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219802 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219804 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219807 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219809 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219812 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219815 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219818 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219821 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219823 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219827 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219829 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219832 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219834 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219837 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:09:50.220439 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219840 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219842 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219845 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219847 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219850 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219856 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219859 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219862 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219866 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219868 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219871 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219874 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219876 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219879 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219882 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219884 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219887 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219889 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.219892 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:09:50.220892 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220617 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220625 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220628 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220631 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220634 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220637 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220640 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220643 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220646 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220648 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220651 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220653 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220656 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220659 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220661 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220666 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220669 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220671 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220674 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:09:50.221360 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220677 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220680 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220682 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220685 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220688 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220691 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220694 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220697 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220700 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220702 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220705 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220708 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220710 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220713 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220716 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220718 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220721 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220723 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220726 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220730 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:09:50.221846 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220733 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220735 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220738 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220740 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220743 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220745 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220748 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220751 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220754 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220757 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220760 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220764 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220769 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220772 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220775 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220777 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220780 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220783 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220786 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220788 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:09:50.222402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220791 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220793 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220796 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220799 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220802 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220804 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220807 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220810 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220812 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220817 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220820 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220824 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220827 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220830 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220833 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220836 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220839 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220842 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220845 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:09:50.222890 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220848 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220850 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220853 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220856 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220859 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220862 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220864 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220867 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220869 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220872 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220874 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220877 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220879 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220882 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220884 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220889 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220893 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220896 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220900 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220903 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:09:50.223363 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.220906 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221352 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221359 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221363 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221366 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221369 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221373 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221380 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221384 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221388 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221391 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221395 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221398 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221401 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221404 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221407 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221410 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221413 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221416 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:09:50.223858 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221418 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221421 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221424 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221427 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221430 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221432 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221436 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221440 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221443 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221446 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221450 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221453 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221456 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221459 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221462 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221465 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221467 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221470 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221473 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:09:50.224327 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221483 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221486 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221489 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221491 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221494 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221497 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221500 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221503 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221506 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221508 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221511 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221514 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221516 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221519 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221521 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221524 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221527 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221529 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221532 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221534 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:09:50.224798 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221537 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221539 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221542 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221544 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221547 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221563 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221568 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221571 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221574 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221578 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221580 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221584 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221587 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221591 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221594 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221596 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221599 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221602 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221606 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221608 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:09:50.225298 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221611 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221613 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221616 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221619 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221621 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221624 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221627 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221629 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221632 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221634 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221637 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221640 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221642 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221645 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221648 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221650 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221653 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221656 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221658 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221661 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:09:50.225779 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221664 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.221672 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221751 2553 flags.go:64] FLAG: --address="0.0.0.0" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221814 2553 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221825 2553 flags.go:64] FLAG: --anonymous-auth="true" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221830 2553 flags.go:64] FLAG: --application-metrics-count-limit="100" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221841 2553 flags.go:64] FLAG: --authentication-token-webhook="false" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221844 2553 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221849 2553 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221853 2553 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221857 2553 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221860 2553 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221863 2553 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221867 2553 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221870 2553 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221873 2553 flags.go:64] FLAG: --cgroup-root="" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221876 2553 flags.go:64] FLAG: --cgroups-per-qos="true" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221879 2553 flags.go:64] FLAG: --client-ca-file="" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221881 2553 flags.go:64] FLAG: --cloud-provider="external" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221884 2553 flags.go:64] FLAG: --cluster-dns="[]" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221889 2553 flags.go:64] FLAG: --cluster-domain="" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221891 2553 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221895 2553 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221898 2553 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Sep 09 21:09:50.226295 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221901 2553 flags.go:64] FLAG: --container-log-max-files="5" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221905 2553 flags.go:64] FLAG: --container-log-max-size="10Mi" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221908 2553 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221911 2553 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221914 2553 flags.go:64] FLAG: --containerd-namespace="k8s.io" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221917 2553 flags.go:64] FLAG: --contention-profiling="false" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221920 2553 flags.go:64] FLAG: --cpu-cfs-quota="true" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221923 2553 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221926 2553 flags.go:64] FLAG: --cpu-manager-policy="none" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221930 2553 flags.go:64] FLAG: --cpu-manager-policy-options="" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221935 2553 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221938 2553 flags.go:64] FLAG: --enable-controller-attach-detach="true" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221941 2553 flags.go:64] FLAG: --enable-debugging-handlers="true" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221944 2553 flags.go:64] FLAG: --enable-load-reader="false" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221948 2553 flags.go:64] FLAG: --enable-server="true" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221958 2553 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221963 2553 flags.go:64] FLAG: --event-burst="100" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221966 2553 flags.go:64] FLAG: --event-qps="50" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221969 2553 flags.go:64] FLAG: --event-storage-age-limit="default=0" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221972 2553 flags.go:64] FLAG: --event-storage-event-limit="default=0" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221975 2553 flags.go:64] FLAG: --eviction-hard="" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221978 2553 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221981 2553 flags.go:64] FLAG: --eviction-minimum-reclaim="" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221984 2553 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221987 2553 flags.go:64] FLAG: --eviction-soft="" Sep 09 21:09:50.226881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221990 2553 flags.go:64] FLAG: --eviction-soft-grace-period="" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221993 2553 flags.go:64] FLAG: --exit-on-lock-contention="false" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221996 2553 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.221999 2553 flags.go:64] FLAG: --experimental-mounter-path="" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222002 2553 flags.go:64] FLAG: --fail-cgroupv1="false" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222005 2553 flags.go:64] FLAG: --fail-swap-on="true" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222008 2553 flags.go:64] FLAG: --feature-gates="" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222011 2553 flags.go:64] FLAG: --file-check-frequency="20s" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222014 2553 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222017 2553 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222021 2553 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222024 2553 flags.go:64] FLAG: --healthz-port="10248" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222027 2553 flags.go:64] FLAG: --help="false" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222030 2553 flags.go:64] FLAG: --hostname-override="ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222033 2553 flags.go:64] FLAG: --housekeeping-interval="10s" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222036 2553 flags.go:64] FLAG: --http-check-frequency="20s" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222041 2553 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222044 2553 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222048 2553 flags.go:64] FLAG: --image-gc-high-threshold="85" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222051 2553 flags.go:64] FLAG: --image-gc-low-threshold="80" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222054 2553 flags.go:64] FLAG: --image-service-endpoint="" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222057 2553 flags.go:64] FLAG: --kernel-memcg-notification="false" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222060 2553 flags.go:64] FLAG: --kube-api-burst="100" Sep 09 21:09:50.227587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222066 2553 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222069 2553 flags.go:64] FLAG: --kube-api-qps="50" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222072 2553 flags.go:64] FLAG: --kube-reserved="" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222075 2553 flags.go:64] FLAG: --kube-reserved-cgroup="" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222078 2553 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222081 2553 flags.go:64] FLAG: --kubelet-cgroups="" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222084 2553 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222087 2553 flags.go:64] FLAG: --lock-file="" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222091 2553 flags.go:64] FLAG: --log-cadvisor-usage="false" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222111 2553 flags.go:64] FLAG: --log-flush-frequency="5s" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222117 2553 flags.go:64] FLAG: --log-json-info-buffer-size="0" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222125 2553 flags.go:64] FLAG: --log-json-split-stream="false" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222129 2553 flags.go:64] FLAG: --log-text-info-buffer-size="0" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222132 2553 flags.go:64] FLAG: --log-text-split-stream="false" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222135 2553 flags.go:64] FLAG: --logging-format="text" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222138 2553 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222142 2553 flags.go:64] FLAG: --make-iptables-util-chains="true" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222144 2553 flags.go:64] FLAG: --manifest-url="" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222147 2553 flags.go:64] FLAG: --manifest-url-header="" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222152 2553 flags.go:64] FLAG: --max-housekeeping-interval="15s" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222155 2553 flags.go:64] FLAG: --max-open-files="1000000" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222159 2553 flags.go:64] FLAG: --max-pods="110" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222162 2553 flags.go:64] FLAG: --maximum-dead-containers="-1" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222165 2553 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222168 2553 flags.go:64] FLAG: --memory-manager-policy="None" Sep 09 21:09:50.228223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222173 2553 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222176 2553 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222182 2553 flags.go:64] FLAG: --node-ip="0.0.0.0" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222185 2553 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222193 2553 flags.go:64] FLAG: --node-status-max-images="50" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222196 2553 flags.go:64] FLAG: --node-status-update-frequency="10s" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222199 2553 flags.go:64] FLAG: --oom-score-adj="-999" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222203 2553 flags.go:64] FLAG: --pod-cidr="" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222222 2553 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222228 2553 flags.go:64] FLAG: --pod-manifest-path="" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222231 2553 flags.go:64] FLAG: --pod-max-pids="-1" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222235 2553 flags.go:64] FLAG: --pods-per-core="0" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222238 2553 flags.go:64] FLAG: --port="10250" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222241 2553 flags.go:64] FLAG: --protect-kernel-defaults="false" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222244 2553 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03b7496010e9613e6" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222247 2553 flags.go:64] FLAG: --qos-reserved="" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222250 2553 flags.go:64] FLAG: --read-only-port="10255" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222254 2553 flags.go:64] FLAG: --register-node="true" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222257 2553 flags.go:64] FLAG: --register-with-taints="" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222260 2553 flags.go:64] FLAG: --registry-burst="10" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222263 2553 flags.go:64] FLAG: --registry-qps="5" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222266 2553 flags.go:64] FLAG: --reserved-cpus="" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222269 2553 flags.go:64] FLAG: --reserved-memory="" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222273 2553 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222276 2553 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Sep 09 21:09:50.228867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222279 2553 flags.go:64] FLAG: --rotate-certificates="false" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222282 2553 flags.go:64] FLAG: --rotate-server-certificates="false" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222285 2553 flags.go:64] FLAG: --runonce="false" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222288 2553 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222291 2553 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222294 2553 flags.go:64] FLAG: --seccomp-default="false" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222297 2553 flags.go:64] FLAG: --serialize-image-pulls="true" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222301 2553 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222305 2553 flags.go:64] FLAG: --storage-driver-db="cadvisor" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222310 2553 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222313 2553 flags.go:64] FLAG: --storage-driver-password="root" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222316 2553 flags.go:64] FLAG: --storage-driver-secure="false" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222319 2553 flags.go:64] FLAG: --storage-driver-table="stats" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222322 2553 flags.go:64] FLAG: --storage-driver-user="root" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222324 2553 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222328 2553 flags.go:64] FLAG: --sync-frequency="1m0s" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222336 2553 flags.go:64] FLAG: --system-cgroups="" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222339 2553 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222345 2553 flags.go:64] FLAG: --system-reserved-cgroup="" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222348 2553 flags.go:64] FLAG: --tls-cert-file="" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222351 2553 flags.go:64] FLAG: --tls-cipher-suites="[]" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222355 2553 flags.go:64] FLAG: --tls-min-version="" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222358 2553 flags.go:64] FLAG: --tls-private-key-file="" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222360 2553 flags.go:64] FLAG: --topology-manager-policy="none" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222363 2553 flags.go:64] FLAG: --topology-manager-policy-options="" Sep 09 21:09:50.229529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222367 2553 flags.go:64] FLAG: --topology-manager-scope="container" Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222370 2553 flags.go:64] FLAG: --v="2" Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222374 2553 flags.go:64] FLAG: --version="false" Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222378 2553 flags.go:64] FLAG: --vmodule="" Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222382 2553 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222386 2553 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222522 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222527 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222530 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222534 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222537 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222542 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222546 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222549 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222552 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222557 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222562 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222565 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222568 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222571 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:09:50.230162 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222574 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222577 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222580 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222582 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222586 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222588 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222591 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222594 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222597 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222599 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222602 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222605 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222607 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222610 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222613 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222615 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222619 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222622 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222625 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222627 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:09:50.230666 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222630 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222634 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222637 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222640 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222643 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222645 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222648 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222652 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222656 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222659 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222662 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222665 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222677 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222680 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222683 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222685 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222688 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222692 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222695 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222698 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:09:50.231182 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222701 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222703 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222706 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222709 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222711 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222714 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222717 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222720 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222722 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222725 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222727 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222730 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222733 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222735 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222738 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222741 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222743 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222746 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222748 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222752 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:09:50.231701 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222756 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222759 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222762 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222764 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222767 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222770 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222772 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222775 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222778 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222781 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222784 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222786 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222789 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222791 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222794 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222797 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222799 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222802 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222805 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222807 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:09:50.232229 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222810 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222813 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222816 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222818 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.222821 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.222830 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.230619 2553 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.230639 2553 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.231548 2553 watchdog_linux.go:95] "Systemd watchdog is not enabled" Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.231557 2553 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231633 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231638 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231642 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231646 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231649 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:09:50.232732 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231652 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231655 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231657 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231660 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231663 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231666 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231669 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231671 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231674 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231677 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231679 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231682 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231684 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231687 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231689 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231692 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231695 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231698 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231700 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231703 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:09:50.233131 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231706 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231708 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231711 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231714 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231722 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231725 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231728 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231731 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231734 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231737 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231739 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231742 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231745 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231747 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231750 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231753 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231756 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231759 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231761 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231764 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:09:50.233634 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231766 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231769 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231772 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231774 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231779 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231783 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231786 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231789 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231793 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231796 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231798 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231801 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231804 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231806 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231809 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231812 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231815 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231818 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231821 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:09:50.234140 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231824 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231826 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231829 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231831 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231834 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231836 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231839 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231841 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231844 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231848 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231852 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231855 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231858 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231861 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231863 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231866 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231869 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231871 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231874 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:09:50.234612 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231878 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231881 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231884 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231887 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231890 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231893 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231896 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231899 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231901 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231904 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231907 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231910 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231913 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231916 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231919 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:09:50.235121 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.231922 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.231927 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232061 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232068 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232071 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232074 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232076 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232080 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232083 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232086 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232088 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232091 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232094 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232112 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232116 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Sep 09 21:09:50.235490 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232118 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232123 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232127 2553 feature_gate.go:328] unrecognized feature gate: Example Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232130 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232133 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232136 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232139 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232142 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232145 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232147 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232150 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232152 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232156 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232159 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232162 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232165 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232168 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232171 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232174 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232176 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Sep 09 21:09:50.235878 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232179 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232182 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232185 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232187 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232190 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232192 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232195 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232198 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232200 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232203 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232205 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232208 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232211 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232213 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232216 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232218 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232222 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232225 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232228 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Sep 09 21:09:50.236402 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232230 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232233 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232236 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232238 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232241 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232245 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232248 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232250 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232253 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232256 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232259 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232261 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232264 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232266 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232269 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232272 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232275 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232277 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232279 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232282 2553 feature_gate.go:328] unrecognized feature gate: Example2 Sep 09 21:09:50.236870 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232284 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232287 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232289 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232292 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232294 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232297 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232299 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232302 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232306 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232309 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232312 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232314 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232317 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232320 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232323 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232326 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232329 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232332 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232334 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Sep 09 21:09:50.237381 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232337 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232339 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232342 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232346 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232348 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232351 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232353 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:50.232356 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.232361 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.233372 2553 server.go:962] "Client rotation is on, will bootstrap in background" Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.235699 2553 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.236757 2553 server.go:1019] "Starting client certificate rotation" Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.236857 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 21:09:50.237840 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.236896 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Sep 09 21:09:50.240081 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.240065 2553 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 21:09:50.242943 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.241650 2553 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Sep 09 21:09:50.256289 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.256126 2553 log.go:25] "Validated CRI v1 runtime API" Sep 09 21:09:50.261226 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.261207 2553 log.go:25] "Validated CRI v1 image API" Sep 09 21:09:50.262110 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.262085 2553 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Sep 09 21:09:50.266272 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.266251 2553 fs.go:135] Filesystem UUIDs: map[338523b0-86d1-4338-b8e2-ad3efead5451:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 96c32ca3-e604-4bd7-870a-3147b4b31388:/dev/nvme0n1p3] Sep 09 21:09:50.266349 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.266270 2553 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Sep 09 21:09:50.272406 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.272286 2553 manager.go:216] Machine: {Timestamp:2026-09-09 21:09:50.269935421 +0000 UTC m=+0.376679249 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094640 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fa34e925359dc03268896f029050c SystemUUID:ec2fa34e-9253-59dc-0326-8896f029050c BootID:d9b3d162-5dc7-4902-b40d-973ca983b661 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:05:6c:c3:27:67 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:05:6c:c3:27:67 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:e6:fa:00:6b:d4 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Sep 09 21:09:50.272406 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.272398 2553 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Sep 09 21:09:50.272532 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.272489 2553 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Sep 09 21:09:50.276228 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.276196 2553 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Sep 09 21:09:50.276379 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.276233 2553 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-115.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Sep 09 21:09:50.276739 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.276724 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 21:09:50.277123 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.277111 2553 topology_manager.go:138] "Creating topology manager with none policy" Sep 09 21:09:50.277164 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.277126 2553 container_manager_linux.go:309] "Creating device plugin manager" Sep 09 21:09:50.277164 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.277134 2553 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 21:09:50.277992 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.277982 2553 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Sep 09 21:09:50.279996 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.279984 2553 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Sep 09 21:09:50.284596 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.284583 2553 state_mem.go:36] "Initialized new in-memory state store" Sep 09 21:09:50.284714 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.284704 2553 server.go:1240] "Using root directory" path="/var/lib/kubelet" Sep 09 21:09:50.284748 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.284736 2553 kubelet.go:486] "Attempting to sync node with API server" Sep 09 21:09:50.284748 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.284746 2553 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Sep 09 21:09:50.284807 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.284758 2553 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Sep 09 21:09:50.284807 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.284767 2553 kubelet.go:398] "Adding apiserver pod source" Sep 09 21:09:50.284807 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.284777 2553 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Sep 09 21:09:50.285168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.285155 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 21:09:50.285168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.285180 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Sep 09 21:09:50.287623 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.287608 2553 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1" Sep 09 21:09:50.290666 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.290652 2553 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Sep 09 21:09:50.290716 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.290683 2553 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Sep 09 21:09:50.290843 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.290833 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Sep 09 21:09:50.290882 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.290847 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Sep 09 21:09:50.290882 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.290854 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Sep 09 21:09:50.290882 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.290859 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Sep 09 21:09:50.290882 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.290867 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Sep 09 21:09:50.290882 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.290873 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Sep 09 21:09:50.291506 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.291496 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Sep 09 21:09:50.291506 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.291505 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Sep 09 21:09:50.291566 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.291512 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Sep 09 21:09:50.291566 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.291518 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Sep 09 21:09:50.291566 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.291535 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Sep 09 21:09:50.291566 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.291542 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Sep 09 21:09:50.293252 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.293241 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Sep 09 21:09:50.293304 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.293257 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Sep 09 21:09:50.297123 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.297094 2553 server.go:1268] "Started kubelet" Sep 09 21:09:50.297205 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.297169 2553 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Sep 09 21:09:50.297347 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.297248 2553 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Sep 09 21:09:50.297465 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.297389 2553 server_v1.go:49] "podresources" method="list" useActivePods=true Sep 09 21:09:50.297520 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.297500 2553 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-115.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Sep 09 21:09:50.297954 ip-10-0-137-115 systemd[1]: Started Kubernetes Kubelet. Sep 09 21:09:50.301581 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.301547 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-115.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Sep 09 21:09:50.301807 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.301782 2553 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Sep 09 21:09:50.301974 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.301954 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Sep 09 21:09:50.303958 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.303910 2553 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Sep 09 21:09:50.304067 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.304012 2553 server.go:310] "Adding debug handlers to kubelet server" Sep 09 21:09:50.308759 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.308735 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Sep 09 21:09:50.308869 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.307695 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-115.ec2.internal.18d3c347d6067de3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-115.ec2.internal,UID:ip-10-0-137-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-115.ec2.internal,},FirstTimestamp:2026-09-09 21:09:50.297062883 +0000 UTC m=+0.403806700,LastTimestamp:2026-09-09 21:09:50.297062883 +0000 UTC m=+0.403806700,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-115.ec2.internal,}" Sep 09 21:09:50.309423 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.309398 2553 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Sep 09 21:09:50.309676 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.309655 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-115.ec2.internal\" not found" Sep 09 21:09:50.310248 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.310229 2553 volume_manager.go:311] "The desired_state_of_world populator starts" Sep 09 21:09:50.310341 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.310260 2553 volume_manager.go:313] "Starting Kubelet Volume Manager" Sep 09 21:09:50.310341 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.310239 2553 factory.go:55] Registering systemd factory Sep 09 21:09:50.310341 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.310281 2553 factory.go:223] Registration of the systemd container factory successfully Sep 09 21:09:50.310341 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.310229 2553 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Sep 09 21:09:50.310499 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.310357 2553 reconstruct.go:97] "Volume reconstruction finished" Sep 09 21:09:50.310499 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.310369 2553 reconciler.go:29] "Reconciler: start to sync state" Sep 09 21:09:50.310599 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.310513 2553 factory.go:153] Registering CRI-O factory Sep 09 21:09:50.310599 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.310541 2553 factory.go:223] Registration of the crio container factory successfully Sep 09 21:09:50.310599 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.310589 2553 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Sep 09 21:09:50.310747 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.310606 2553 factory.go:103] Registering Raw factory Sep 09 21:09:50.310747 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.310621 2553 manager.go:1196] Started watching for new ooms in manager Sep 09 21:09:50.313064 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.313047 2553 manager.go:349] Starting recovery of all containers Sep 09 21:09:50.317883 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.317856 2553 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-115.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Sep 09 21:09:50.318020 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.317967 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Sep 09 21:09:50.318788 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.318750 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Sep 09 21:09:50.322146 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.322071 2553 manager.go:354] Recovery completed Sep 09 21:09:50.323792 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.323763 2553 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Sep 09 21:09:50.326842 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.326828 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:50.328663 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.328647 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:50.328729 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.328676 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:50.328729 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.328686 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:50.329371 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.329356 2553 cpu_manager.go:222] "Starting CPU manager" policy="none" Sep 09 21:09:50.329371 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.329368 2553 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Sep 09 21:09:50.329488 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.329387 2553 state_mem.go:36] "Initialized new in-memory state store" Sep 09 21:09:50.329488 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.329360 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:50.330165 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.330151 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:50.330228 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.330181 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:50.330228 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.330194 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:50.331419 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.331344 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-115.ec2.internal.18d3c347d7e8abee default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-115.ec2.internal,UID:ip-10-0-137-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-115.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-115.ec2.internal,},FirstTimestamp:2026-09-09 21:09:50.328663022 +0000 UTC m=+0.435406839,LastTimestamp:2026-09-09 21:09:50.328663022 +0000 UTC m=+0.435406839,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-115.ec2.internal,}" Sep 09 21:09:50.331496 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.331484 2553 policy_none.go:49] "None policy: Start" Sep 09 21:09:50.331526 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.331516 2553 memory_manager.go:187] "Starting memorymanager" policy="None" Sep 09 21:09:50.331567 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.331532 2553 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Sep 09 21:09:50.333531 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.333518 2553 policy_none.go:47] "Start" Sep 09 21:09:50.334174 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.334154 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ptttn" Sep 09 21:09:50.340260 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.340192 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-115.ec2.internal.18d3c347d7e8f36f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-115.ec2.internal,UID:ip-10-0-137-115.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-115.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-115.ec2.internal,},FirstTimestamp:2026-09-09 21:09:50.328681327 +0000 UTC m=+0.435425143,LastTimestamp:2026-09-09 21:09:50.328681327 +0000 UTC m=+0.435425143,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-115.ec2.internal,}" Sep 09 21:09:50.345375 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.345356 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ptttn" Sep 09 21:09:50.376744 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.376725 2553 manager.go:337] "Starting Device Plugin manager" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.376776 2553 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.376790 2553 server.go:87] "Starting device plugin registration server" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.377062 2553 eviction_manager.go:189] "Eviction manager: starting control loop" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.377073 2553 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.377176 2553 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.377271 2553 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.377279 2553 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.377900 2553 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.377965 2553 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-115.ec2.internal\" not found" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.382761 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.382807 2553 status_manager.go:244] "Starting to sync pod status with apiserver" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.382827 2553 kubelet.go:2448] "Starting kubelet main sync loop" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.382859 2553 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Sep 09 21:09:50.392131 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.385221 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:09:50.477636 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.477544 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:50.478604 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.478589 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:50.478662 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.478620 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:50.478662 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.478631 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:50.478662 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.478656 2553 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.483722 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.483704 2553 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-115.ec2.internal"] Sep 09 21:09:50.483782 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.483773 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:50.484607 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.484592 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:50.484681 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.484617 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:50.484681 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.484627 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:50.486906 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.486893 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:50.487037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.487008 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.487037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.487041 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:50.487598 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.487583 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:50.487598 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.487590 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:50.487744 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.487607 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:50.487744 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.487616 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:50.487744 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.487620 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:50.487744 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.487631 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:50.487908 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.487819 2553 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.487908 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.487834 2553 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-115.ec2.internal\": node \"ip-10-0-137-115.ec2.internal\" not found" Sep 09 21:09:50.489885 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.489869 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.489967 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.489900 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:50.490652 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.490638 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:50.490726 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.490661 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:50.490726 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.490674 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:50.511616 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.511580 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8edc6fae6fa3feede68fb3f030e50c39-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal\" (UID: \"8edc6fae6fa3feede68fb3f030e50c39\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.511772 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.511620 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8edc6fae6fa3feede68fb3f030e50c39-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal\" (UID: \"8edc6fae6fa3feede68fb3f030e50c39\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.511772 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.511645 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0531880037de699c667aef422549784e-config\") pod \"kube-apiserver-proxy-ip-10-0-137-115.ec2.internal\" (UID: \"0531880037de699c667aef422549784e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.516021 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.516003 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-115.ec2.internal\" not found" node="ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.518983 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.518964 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-115.ec2.internal\" not found" Sep 09 21:09:50.520453 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.520436 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-115.ec2.internal\" not found" node="ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.612750 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.612710 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8edc6fae6fa3feede68fb3f030e50c39-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal\" (UID: \"8edc6fae6fa3feede68fb3f030e50c39\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.612750 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.612748 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8edc6fae6fa3feede68fb3f030e50c39-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal\" (UID: \"8edc6fae6fa3feede68fb3f030e50c39\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.612966 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.612768 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0531880037de699c667aef422549784e-config\") pod \"kube-apiserver-proxy-ip-10-0-137-115.ec2.internal\" (UID: \"0531880037de699c667aef422549784e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.612966 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.612818 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0531880037de699c667aef422549784e-config\") pod \"kube-apiserver-proxy-ip-10-0-137-115.ec2.internal\" (UID: \"0531880037de699c667aef422549784e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.612966 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.612823 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8edc6fae6fa3feede68fb3f030e50c39-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal\" (UID: \"8edc6fae6fa3feede68fb3f030e50c39\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.612966 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.612835 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8edc6fae6fa3feede68fb3f030e50c39-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal\" (UID: \"8edc6fae6fa3feede68fb3f030e50c39\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.619028 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.619008 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-115.ec2.internal\" not found" Sep 09 21:09:50.719232 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.719190 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-115.ec2.internal\" not found" Sep 09 21:09:50.820402 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.820319 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-115.ec2.internal\" not found" Sep 09 21:09:50.820402 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.820362 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.824448 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:50.824426 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-115.ec2.internal" Sep 09 21:09:50.920595 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:50.920551 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-115.ec2.internal\" not found" Sep 09 21:09:51.020846 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:51.020817 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-115.ec2.internal\" not found" Sep 09 21:09:51.121040 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:51.120944 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-115.ec2.internal\" not found" Sep 09 21:09:51.221233 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:51.221192 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-115.ec2.internal\" not found" Sep 09 21:09:51.236484 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.236456 2553 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Sep 09 21:09:51.236646 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.236623 2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Sep 09 21:09:51.313589 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.313509 2553 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:09:51.321824 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:51.321793 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-115.ec2.internal\" not found" Sep 09 21:09:51.347978 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.347926 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 21:04:50 +0000 UTC" deadline="2028-04-29 22:47:59.217895185 +0000 UTC" Sep 09 21:09:51.348139 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.347974 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14353h38m7.869925645s" Sep 09 21:09:51.365434 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.365406 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Sep 09 21:09:51.366840 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.366814 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientMemory" Sep 09 21:09:51.366952 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.366858 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasNoDiskPressure" Sep 09 21:09:51.366952 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.366870 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeHasSufficientPID" Sep 09 21:09:51.375906 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.375844 2553 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:09:51.410654 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.410615 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" Sep 09 21:09:51.427903 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.427876 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 21:09:51.429979 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.429962 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-115.ec2.internal" Sep 09 21:09:51.435216 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:51.435170 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0531880037de699c667aef422549784e.slice/crio-3aa4af91292eb8f071a72fa650b50a77335153b6a3ae41585721db3a34ae4f42 WatchSource:0}: Error finding container 3aa4af91292eb8f071a72fa650b50a77335153b6a3ae41585721db3a34ae4f42: Status 404 returned error can't find the container with id 3aa4af91292eb8f071a72fa650b50a77335153b6a3ae41585721db3a34ae4f42 Sep 09 21:09:51.436034 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:51.436016 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8edc6fae6fa3feede68fb3f030e50c39.slice/crio-81fc016a3080cd23e95de62f401330baa1eeab6aab5292dac10b89f6e990d10c WatchSource:0}: Error finding container 81fc016a3080cd23e95de62f401330baa1eeab6aab5292dac10b89f6e990d10c: Status 404 returned error can't find the container with id 81fc016a3080cd23e95de62f401330baa1eeab6aab5292dac10b89f6e990d10c Sep 09 21:09:51.440977 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.440895 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Sep 09 21:09:51.441528 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.441510 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:09:51.667972 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:51.667870 2553 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:09:52.110930 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.110691 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Sep 09 21:09:52.286919 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.286882 2553 apiserver.go:52] "Watching apiserver" Sep 09 21:09:52.294226 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.294189 2553 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Sep 09 21:09:52.294701 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.294668 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-nxm74","openshift-network-diagnostics/network-check-target-vk8h4","kube-system/konnectivity-agent-bf4gk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf","openshift-dns/node-resolver-sz8l8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal","openshift-multus/multus-j4r27","openshift-multus/network-metrics-daemon-55b8r","openshift-network-operator/iptables-alerter-6kv6p","openshift-ovn-kubernetes/ovnkube-node-zmmsz","kube-system/global-pull-secret-syncer-tbgn7","kube-system/kube-apiserver-proxy-ip-10-0-137-115.ec2.internal","openshift-cluster-node-tuning-operator/tuned-57dnk","openshift-image-registry/node-ca-srt4v"] Sep 09 21:09:52.297567 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.297544 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.299948 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.299921 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:09:52.300056 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:52.300014 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:09:52.300582 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.300530 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Sep 09 21:09:52.300582 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.300562 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Sep 09 21:09:52.300859 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.300590 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Sep 09 21:09:52.300940 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.300862 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Sep 09 21:09:52.301114 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.301090 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Sep 09 21:09:52.301215 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.301092 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r2mr6\"" Sep 09 21:09:52.301276 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.301224 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Sep 09 21:09:52.304548 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.304249 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.305123 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.305101 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.307591 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.307252 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2l7cg\"" Sep 09 21:09:52.307591 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.307449 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Sep 09 21:09:52.308796 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.307794 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Sep 09 21:09:52.308796 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.308026 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Sep 09 21:09:52.308796 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.308258 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Sep 09 21:09:52.308796 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.308270 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Sep 09 21:09:52.308796 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.308491 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sz8l8" Sep 09 21:09:52.308796 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.308498 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Sep 09 21:09:52.308796 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.308567 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Sep 09 21:09:52.308796 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.308601 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Sep 09 21:09:52.308796 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.308703 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4fshk\"" Sep 09 21:09:52.309667 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.309509 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Sep 09 21:09:52.311710 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.311163 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Sep 09 21:09:52.311710 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.311533 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wczkl\"" Sep 09 21:09:52.311864 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.311429 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Sep 09 21:09:52.311925 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.311910 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.314195 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.314018 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:09:52.314195 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:52.314093 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:09:52.314556 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.314515 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Sep 09 21:09:52.314654 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.314626 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-57m4h\"" Sep 09 21:09:52.316867 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.316847 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6kv6p" Sep 09 21:09:52.320343 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.320294 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Sep 09 21:09:52.320582 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.320566 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:09:52.320669 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.320610 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Sep 09 21:09:52.321409 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.321388 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fl4ll\"" Sep 09 21:09:52.322210 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.321876 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bf4gk" Sep 09 21:09:52.324630 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324595 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-var-lib-kubelet\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.324630 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324627 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-hostroot\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.324794 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324655 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-var-lib-cni-multus\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.324794 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324683 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-run-multus-certs\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.324794 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324706 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:52.324794 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324729 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xh4r9\"" Sep 09 21:09:52.324794 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324706 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-etc-selinux\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.324794 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:52.324770 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:09:52.324794 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324783 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91388845-f81d-42fa-b7ec-5e50730971ea-hosts-file\") pod \"node-resolver-sz8l8\" (UID: \"91388845-f81d-42fa-b7ec-5e50730971ea\") " pod="openshift-dns/node-resolver-sz8l8" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324806 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-etc-kubernetes\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324829 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324853 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn9h7\" (UniqueName: \"kubernetes.io/projected/d70094ac-92b5-4cfd-bad5-522da65bbf9e-kube-api-access-dn9h7\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324878 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl58d\" (UniqueName: \"kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d\") pod \"network-check-target-vk8h4\" (UID: \"7ed0ff95-e5da-49df-b563-a0988e9e8fe3\") " pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324903 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b72de97-ed02-4e23-84db-3d6a80a3e685-cni-binary-copy\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324930 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-cnibin\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324973 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-cni-bin\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324990 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.324998 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-device-dir\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325024 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b72de97-ed02-4e23-84db-3d6a80a3e685-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325049 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-system-cni-dir\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325075 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-var-lib-cni-bin\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325103 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b72de97-ed02-4e23-84db-3d6a80a3e685-system-cni-dir\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325128 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-run-k8s-cni-cncf-io\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.325168 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325131 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325197 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-log-socket\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325250 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d70094ac-92b5-4cfd-bad5-522da65bbf9e-ovnkube-config\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325280 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b72de97-ed02-4e23-84db-3d6a80a3e685-cnibin\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325325 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b72de97-ed02-4e23-84db-3d6a80a3e685-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325408 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-run-systemd\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325490 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-etc-openvswitch\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325515 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91388845-f81d-42fa-b7ec-5e50730971ea-tmp-dir\") pod \"node-resolver-sz8l8\" (UID: \"91388845-f81d-42fa-b7ec-5e50730971ea\") " pod="openshift-dns/node-resolver-sz8l8" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325539 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0d6e1e2-96f1-4c46-b850-43586ded2151-multus-daemon-config\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325563 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-multus-socket-dir-parent\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325587 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-run-ovn\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325623 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0d6e1e2-96f1-4c46-b850-43586ded2151-cni-binary-copy\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325655 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-multus-conf-dir\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325691 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-run-netns\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325718 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d70094ac-92b5-4cfd-bad5-522da65bbf9e-ovnkube-script-lib\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325747 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-sys-fs\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.325973 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325790 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvvhm\" (UniqueName: \"kubernetes.io/projected/74a4c44e-98e5-4128-aee2-84987c8f66a4-kube-api-access-wvvhm\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325818 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcnxt\" (UniqueName: \"kubernetes.io/projected/e0d6e1e2-96f1-4c46-b850-43586ded2151-kube-api-access-jcnxt\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325843 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-run-openvswitch\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325869 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-cni-netd\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325897 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d70094ac-92b5-4cfd-bad5-522da65bbf9e-ovn-node-metrics-cert\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325924 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-registration-dir\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325951 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-node-log\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.325977 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b72de97-ed02-4e23-84db-3d6a80a3e685-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326004 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4xmb\" (UniqueName: \"kubernetes.io/projected/91388845-f81d-42fa-b7ec-5e50730971ea-kube-api-access-b4xmb\") pod \"node-resolver-sz8l8\" (UID: \"91388845-f81d-42fa-b7ec-5e50730971ea\") " pod="openshift-dns/node-resolver-sz8l8" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326032 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-multus-cni-dir\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326057 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-kubelet\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326084 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-run-ovn-kubernetes\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326109 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-systemd-units\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326135 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d70094ac-92b5-4cfd-bad5-522da65bbf9e-env-overrides\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326182 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326209 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnjxc\" (UniqueName: \"kubernetes.io/projected/8b72de97-ed02-4e23-84db-3d6a80a3e685-kube-api-access-rnjxc\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.326778 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326272 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-run-netns\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.327564 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326297 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-slash\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.327564 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326322 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-var-lib-openvswitch\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.327564 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326347 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-socket-dir\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.327564 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.326755 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Sep 09 21:09:52.327564 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.327314 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.330105 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.329872 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-srt4v" Sep 09 21:09:52.330105 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.329990 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:09:52.330331 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.330312 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Sep 09 21:09:52.330711 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.330691 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9gvrx\"" Sep 09 21:09:52.332896 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.332871 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Sep 09 21:09:52.333297 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.333268 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Sep 09 21:09:52.333670 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.333646 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Sep 09 21:09:52.333868 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.333852 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-frb82\"" Sep 09 21:09:52.351233 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.351198 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gg5dx" Sep 09 21:09:52.361032 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.360964 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gg5dx" Sep 09 21:09:52.387800 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.387744 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" event={"ID":"8edc6fae6fa3feede68fb3f030e50c39","Type":"ContainerStarted","Data":"81fc016a3080cd23e95de62f401330baa1eeab6aab5292dac10b89f6e990d10c"} Sep 09 21:09:52.389033 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.389002 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-115.ec2.internal" event={"ID":"0531880037de699c667aef422549784e","Type":"ContainerStarted","Data":"3aa4af91292eb8f071a72fa650b50a77335153b6a3ae41585721db3a34ae4f42"} Sep 09 21:09:52.412991 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.412953 2553 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Sep 09 21:09:52.427383 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427346 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0d6e1e2-96f1-4c46-b850-43586ded2151-cni-binary-copy\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.427383 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427394 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-multus-conf-dir\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.427642 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427421 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-run-netns\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.427642 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427445 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d70094ac-92b5-4cfd-bad5-522da65bbf9e-ovnkube-script-lib\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.427642 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427483 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-multus-conf-dir\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.427642 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427516 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-run-netns\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.427642 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427531 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-sys-fs\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.427642 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427579 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvvhm\" (UniqueName: \"kubernetes.io/projected/74a4c44e-98e5-4128-aee2-84987c8f66a4-kube-api-access-wvvhm\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.427642 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427618 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-sysconfig\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.427642 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427643 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/436fa819-8e30-43d8-ae10-7c9d0cbc04c5-host\") pod \"node-ca-srt4v\" (UID: \"436fa819-8e30-43d8-ae10-7c9d0cbc04c5\") " pod="openshift-image-registry/node-ca-srt4v" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427671 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcnxt\" (UniqueName: \"kubernetes.io/projected/e0d6e1e2-96f1-4c46-b850-43586ded2151-kube-api-access-jcnxt\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427583 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-sys-fs\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427725 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-run-openvswitch\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427899 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-run-openvswitch\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427913 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-cni-netd\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427948 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d70094ac-92b5-4cfd-bad5-522da65bbf9e-ovn-node-metrics-cert\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427975 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-registration-dir\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.427991 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-cni-netd\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428005 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73b17a8d-a244-4ce5-bc9a-d79994d435d7-host-slash\") pod \"iptables-alerter-6kv6p\" (UID: \"73b17a8d-a244-4ce5-bc9a-d79994d435d7\") " pod="openshift-network-operator/iptables-alerter-6kv6p" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428034 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-kubernetes\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428065 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e0d6e1e2-96f1-4c46-b850-43586ded2151-cni-binary-copy\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428070 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7r6gh\" (UniqueName: \"kubernetes.io/projected/436fa819-8e30-43d8-ae10-7c9d0cbc04c5-kube-api-access-7r6gh\") pod \"node-ca-srt4v\" (UID: \"436fa819-8e30-43d8-ae10-7c9d0cbc04c5\") " pod="openshift-image-registry/node-ca-srt4v" Sep 09 21:09:52.428098 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428095 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d70094ac-92b5-4cfd-bad5-522da65bbf9e-ovnkube-script-lib\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428124 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-node-log\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428140 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-registration-dir\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428175 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b72de97-ed02-4e23-84db-3d6a80a3e685-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428205 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-node-log\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428248 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4xmb\" (UniqueName: \"kubernetes.io/projected/91388845-f81d-42fa-b7ec-5e50730971ea-kube-api-access-b4xmb\") pod \"node-resolver-sz8l8\" (UID: \"91388845-f81d-42fa-b7ec-5e50730971ea\") " pod="openshift-dns/node-resolver-sz8l8" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428431 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-run\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428472 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-multus-cni-dir\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428534 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-kubelet\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428562 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-run-ovn-kubernetes\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428586 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-sysctl-conf\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428609 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-systemd-units\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428666 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d70094ac-92b5-4cfd-bad5-522da65bbf9e-env-overrides\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428692 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.428706 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428709 2553 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428718 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnjxc\" (UniqueName: \"kubernetes.io/projected/8b72de97-ed02-4e23-84db-3d6a80a3e685-kube-api-access-rnjxc\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428745 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-run-netns\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428768 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-slash\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428774 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8b72de97-ed02-4e23-84db-3d6a80a3e685-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428810 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-var-lib-openvswitch\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428859 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-socket-dir\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428876 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-kubelet\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428881 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/436fa819-8e30-43d8-ae10-7c9d0cbc04c5-serviceca\") pod \"node-ca-srt4v\" (UID: \"436fa819-8e30-43d8-ae10-7c9d0cbc04c5\") " pod="openshift-image-registry/node-ca-srt4v" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428902 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-var-lib-kubelet\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428907 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-run-netns\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428918 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-hostroot\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428935 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-host\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428952 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-run-ovn-kubernetes\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.428969 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429004 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a77a1afc-9eda-42b4-857c-2967b6d149d8-dbus\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429015 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-systemd-units\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.429337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429028 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429047 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-var-lib-cni-multus\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429057 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-var-lib-openvswitch\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429064 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-run-multus-certs\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429089 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-etc-selinux\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429116 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91388845-f81d-42fa-b7ec-5e50730971ea-hosts-file\") pod \"node-resolver-sz8l8\" (UID: \"91388845-f81d-42fa-b7ec-5e50730971ea\") " pod="openshift-dns/node-resolver-sz8l8" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429161 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgl4s\" (UniqueName: \"kubernetes.io/projected/1abf2101-20e2-4e99-bb20-206110f86504-kube-api-access-kgl4s\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429209 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-slash\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429248 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-var-lib-kubelet\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429263 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91388845-f81d-42fa-b7ec-5e50730971ea-hosts-file\") pod \"node-resolver-sz8l8\" (UID: \"91388845-f81d-42fa-b7ec-5e50730971ea\") " pod="openshift-dns/node-resolver-sz8l8" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429213 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/73b17a8d-a244-4ce5-bc9a-d79994d435d7-iptables-alerter-script\") pod \"iptables-alerter-6kv6p\" (UID: \"73b17a8d-a244-4ce5-bc9a-d79994d435d7\") " pod="openshift-network-operator/iptables-alerter-6kv6p" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429289 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-var-lib-cni-multus\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429302 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4e4311b-1130-4e07-8052-599cbb8ef158-konnectivity-ca\") pod \"konnectivity-agent-bf4gk\" (UID: \"f4e4311b-1130-4e07-8052-599cbb8ef158\") " pod="kube-system/konnectivity-agent-bf4gk" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429324 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-run-multus-certs\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429249 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-multus-cni-dir\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429340 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-socket-dir\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429336 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-etc-kubernetes\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.430034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429384 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-hostroot\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429383 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429416 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429420 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4e4311b-1130-4e07-8052-599cbb8ef158-agent-certs\") pod \"konnectivity-agent-bf4gk\" (UID: \"f4e4311b-1130-4e07-8052-599cbb8ef158\") " pod="kube-system/konnectivity-agent-bf4gk" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429433 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-etc-selinux\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429448 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-modprobe-d\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429465 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d70094ac-92b5-4cfd-bad5-522da65bbf9e-env-overrides\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429475 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jblhz\" (UniqueName: \"kubernetes.io/projected/96490314-f8b7-439d-86e4-f61142aa887f-kube-api-access-jblhz\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429478 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-etc-kubernetes\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429505 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn9h7\" (UniqueName: \"kubernetes.io/projected/d70094ac-92b5-4cfd-bad5-522da65bbf9e-kube-api-access-dn9h7\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429547 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl58d\" (UniqueName: \"kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d\") pod \"network-check-target-vk8h4\" (UID: \"7ed0ff95-e5da-49df-b563-a0988e9e8fe3\") " pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429569 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b72de97-ed02-4e23-84db-3d6a80a3e685-cni-binary-copy\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429595 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96490314-f8b7-439d-86e4-f61142aa887f-tmp\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429619 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a77a1afc-9eda-42b4-857c-2967b6d149d8-kubelet-config\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429658 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-cnibin\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429680 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-cni-bin\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429702 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-device-dir\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.430812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429765 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b72de97-ed02-4e23-84db-3d6a80a3e685-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429787 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-system-cni-dir\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429811 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-var-lib-cni-bin\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429838 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b72de97-ed02-4e23-84db-3d6a80a3e685-system-cni-dir\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429862 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429910 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/74a4c44e-98e5-4128-aee2-84987c8f66a4-device-dir\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429957 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-run-k8s-cni-cncf-io\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429976 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-system-cni-dir\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.429988 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-log-socket\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430011 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d70094ac-92b5-4cfd-bad5-522da65bbf9e-ovnkube-config\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430025 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b72de97-ed02-4e23-84db-3d6a80a3e685-system-cni-dir\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430034 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b72de97-ed02-4e23-84db-3d6a80a3e685-cnibin\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430076 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-cnibin\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430088 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b72de97-ed02-4e23-84db-3d6a80a3e685-cni-binary-copy\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430097 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-var-lib-cni-bin\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430134 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-host-run-k8s-cni-cncf-io\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430156 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-log-socket\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.431524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430189 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b72de97-ed02-4e23-84db-3d6a80a3e685-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430212 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-host-cni-bin\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430218 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-sysctl-d\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430266 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-sys\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430292 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-run-systemd\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430315 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-etc-openvswitch\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430359 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91388845-f81d-42fa-b7ec-5e50730971ea-tmp-dir\") pod \"node-resolver-sz8l8\" (UID: \"91388845-f81d-42fa-b7ec-5e50730971ea\") " pod="openshift-dns/node-resolver-sz8l8" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430375 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b72de97-ed02-4e23-84db-3d6a80a3e685-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430381 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-var-lib-kubelet\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430423 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0d6e1e2-96f1-4c46-b850-43586ded2151-multus-daemon-config\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430426 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-etc-openvswitch\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430446 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmwbk\" (UniqueName: \"kubernetes.io/projected/73b17a8d-a244-4ce5-bc9a-d79994d435d7-kube-api-access-xmwbk\") pod \"iptables-alerter-6kv6p\" (UID: \"73b17a8d-a244-4ce5-bc9a-d79994d435d7\") " pod="openshift-network-operator/iptables-alerter-6kv6p" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430486 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/96490314-f8b7-439d-86e4-f61142aa887f-etc-tuned\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430515 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-multus-socket-dir-parent\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430538 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-run-ovn\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430577 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-systemd\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430608 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8b72de97-ed02-4e23-84db-3d6a80a3e685-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.432096 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430155 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b72de97-ed02-4e23-84db-3d6a80a3e685-cnibin\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.432680 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430651 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-lib-modules\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.432680 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430736 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91388845-f81d-42fa-b7ec-5e50730971ea-tmp-dir\") pod \"node-resolver-sz8l8\" (UID: \"91388845-f81d-42fa-b7ec-5e50730971ea\") " pod="openshift-dns/node-resolver-sz8l8" Sep 09 21:09:52.432680 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430743 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e0d6e1e2-96f1-4c46-b850-43586ded2151-multus-socket-dir-parent\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.432680 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430783 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-run-ovn\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.432680 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.430826 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d70094ac-92b5-4cfd-bad5-522da65bbf9e-run-systemd\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.432680 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.431025 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e0d6e1e2-96f1-4c46-b850-43586ded2151-multus-daemon-config\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.432680 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.431229 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d70094ac-92b5-4cfd-bad5-522da65bbf9e-ovnkube-config\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.433546 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.433517 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d70094ac-92b5-4cfd-bad5-522da65bbf9e-ovn-node-metrics-cert\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.436702 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.436634 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvvhm\" (UniqueName: \"kubernetes.io/projected/74a4c44e-98e5-4128-aee2-84987c8f66a4-kube-api-access-wvvhm\") pod \"aws-ebs-csi-driver-node-r46bf\" (UID: \"74a4c44e-98e5-4128-aee2-84987c8f66a4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.436866 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.436734 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcnxt\" (UniqueName: \"kubernetes.io/projected/e0d6e1e2-96f1-4c46-b850-43586ded2151-kube-api-access-jcnxt\") pod \"multus-j4r27\" (UID: \"e0d6e1e2-96f1-4c46-b850-43586ded2151\") " pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.440580 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:52.440555 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:09:52.440704 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:52.440586 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:09:52.440704 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:52.440601 2553 projected.go:196] Error preparing data for projected volume kube-api-access-wl58d for pod openshift-network-diagnostics/network-check-target-vk8h4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:09:52.440704 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:52.440683 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d podName:7ed0ff95-e5da-49df-b563-a0988e9e8fe3 nodeName:}" failed. No retries permitted until 2026-09-09 21:09:52.940662406 +0000 UTC m=+3.047406214 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wl58d" (UniqueName: "kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d") pod "network-check-target-vk8h4" (UID: "7ed0ff95-e5da-49df-b563-a0988e9e8fe3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:09:52.440905 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.440750 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4xmb\" (UniqueName: \"kubernetes.io/projected/91388845-f81d-42fa-b7ec-5e50730971ea-kube-api-access-b4xmb\") pod \"node-resolver-sz8l8\" (UID: \"91388845-f81d-42fa-b7ec-5e50730971ea\") " pod="openshift-dns/node-resolver-sz8l8" Sep 09 21:09:52.441248 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.441163 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnjxc\" (UniqueName: \"kubernetes.io/projected/8b72de97-ed02-4e23-84db-3d6a80a3e685-kube-api-access-rnjxc\") pod \"multus-additional-cni-plugins-nxm74\" (UID: \"8b72de97-ed02-4e23-84db-3d6a80a3e685\") " pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.443241 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.443216 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn9h7\" (UniqueName: \"kubernetes.io/projected/d70094ac-92b5-4cfd-bad5-522da65bbf9e-kube-api-access-dn9h7\") pod \"ovnkube-node-zmmsz\" (UID: \"d70094ac-92b5-4cfd-bad5-522da65bbf9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.530964 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.530910 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/436fa819-8e30-43d8-ae10-7c9d0cbc04c5-serviceca\") pod \"node-ca-srt4v\" (UID: \"436fa819-8e30-43d8-ae10-7c9d0cbc04c5\") " pod="openshift-image-registry/node-ca-srt4v" Sep 09 21:09:52.530964 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.530962 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-host\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.531251 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.530987 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a77a1afc-9eda-42b4-857c-2967b6d149d8-dbus\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:52.531251 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531016 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:52.531251 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531047 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgl4s\" (UniqueName: \"kubernetes.io/projected/1abf2101-20e2-4e99-bb20-206110f86504-kube-api-access-kgl4s\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:09:52.531251 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531064 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/73b17a8d-a244-4ce5-bc9a-d79994d435d7-iptables-alerter-script\") pod \"iptables-alerter-6kv6p\" (UID: \"73b17a8d-a244-4ce5-bc9a-d79994d435d7\") " pod="openshift-network-operator/iptables-alerter-6kv6p" Sep 09 21:09:52.531251 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531064 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-host\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.531251 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531093 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4e4311b-1130-4e07-8052-599cbb8ef158-konnectivity-ca\") pod \"konnectivity-agent-bf4gk\" (UID: \"f4e4311b-1130-4e07-8052-599cbb8ef158\") " pod="kube-system/konnectivity-agent-bf4gk" Sep 09 21:09:52.531251 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531121 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4e4311b-1130-4e07-8052-599cbb8ef158-agent-certs\") pod \"konnectivity-agent-bf4gk\" (UID: \"f4e4311b-1130-4e07-8052-599cbb8ef158\") " pod="kube-system/konnectivity-agent-bf4gk" Sep 09 21:09:52.531251 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531157 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-modprobe-d\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.531251 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:52.531198 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:09:52.531251 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531223 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jblhz\" (UniqueName: \"kubernetes.io/projected/96490314-f8b7-439d-86e4-f61142aa887f-kube-api-access-jblhz\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531259 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a77a1afc-9eda-42b4-857c-2967b6d149d8-dbus\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531269 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-modprobe-d\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:52.531280 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret podName:a77a1afc-9eda-42b4-857c-2967b6d149d8 nodeName:}" failed. No retries permitted until 2026-09-09 21:09:53.031258119 +0000 UTC m=+3.138001949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret") pod "global-pull-secret-syncer-tbgn7" (UID: "a77a1afc-9eda-42b4-857c-2967b6d149d8") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531320 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96490314-f8b7-439d-86e4-f61142aa887f-tmp\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531348 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a77a1afc-9eda-42b4-857c-2967b6d149d8-kubelet-config\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531383 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531416 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-sysctl-d\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531441 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-sys\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531469 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-var-lib-kubelet\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531499 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmwbk\" (UniqueName: \"kubernetes.io/projected/73b17a8d-a244-4ce5-bc9a-d79994d435d7-kube-api-access-xmwbk\") pod \"iptables-alerter-6kv6p\" (UID: \"73b17a8d-a244-4ce5-bc9a-d79994d435d7\") " pod="openshift-network-operator/iptables-alerter-6kv6p" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531529 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/96490314-f8b7-439d-86e4-f61142aa887f-etc-tuned\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:52.531537 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531541 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/436fa819-8e30-43d8-ae10-7c9d0cbc04c5-serviceca\") pod \"node-ca-srt4v\" (UID: \"436fa819-8e30-43d8-ae10-7c9d0cbc04c5\") " pod="openshift-image-registry/node-ca-srt4v" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531557 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-systemd\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:52.531591 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs podName:1abf2101-20e2-4e99-bb20-206110f86504 nodeName:}" failed. No retries permitted until 2026-09-09 21:09:53.03157148 +0000 UTC m=+3.138315291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs") pod "network-metrics-daemon-55b8r" (UID: "1abf2101-20e2-4e99-bb20-206110f86504") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:09:52.531758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531613 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-lib-modules\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531628 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-var-lib-kubelet\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531651 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-sysconfig\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531674 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/436fa819-8e30-43d8-ae10-7c9d0cbc04c5-host\") pod \"node-ca-srt4v\" (UID: \"436fa819-8e30-43d8-ae10-7c9d0cbc04c5\") " pod="openshift-image-registry/node-ca-srt4v" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531713 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73b17a8d-a244-4ce5-bc9a-d79994d435d7-host-slash\") pod \"iptables-alerter-6kv6p\" (UID: \"73b17a8d-a244-4ce5-bc9a-d79994d435d7\") " pod="openshift-network-operator/iptables-alerter-6kv6p" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531741 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-kubernetes\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531742 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-sysctl-d\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531766 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7r6gh\" (UniqueName: \"kubernetes.io/projected/436fa819-8e30-43d8-ae10-7c9d0cbc04c5-kube-api-access-7r6gh\") pod \"node-ca-srt4v\" (UID: \"436fa819-8e30-43d8-ae10-7c9d0cbc04c5\") " pod="openshift-image-registry/node-ca-srt4v" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531792 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-run\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531802 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/73b17a8d-a244-4ce5-bc9a-d79994d435d7-iptables-alerter-script\") pod \"iptables-alerter-6kv6p\" (UID: \"73b17a8d-a244-4ce5-bc9a-d79994d435d7\") " pod="openshift-network-operator/iptables-alerter-6kv6p" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531818 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-sysctl-conf\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531849 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f4e4311b-1130-4e07-8052-599cbb8ef158-konnectivity-ca\") pod \"konnectivity-agent-bf4gk\" (UID: \"f4e4311b-1130-4e07-8052-599cbb8ef158\") " pod="kube-system/konnectivity-agent-bf4gk" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531866 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-sysconfig\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531913 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a77a1afc-9eda-42b4-857c-2967b6d149d8-kubelet-config\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531964 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-sys\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531976 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-sysctl-conf\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.532013 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/436fa819-8e30-43d8-ae10-7c9d0cbc04c5-host\") pod \"node-ca-srt4v\" (UID: \"436fa819-8e30-43d8-ae10-7c9d0cbc04c5\") " pod="openshift-image-registry/node-ca-srt4v" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.532022 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-lib-modules\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.532590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.532030 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-kubernetes\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.533567 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.531616 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-etc-systemd\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.533567 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.532067 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73b17a8d-a244-4ce5-bc9a-d79994d435d7-host-slash\") pod \"iptables-alerter-6kv6p\" (UID: \"73b17a8d-a244-4ce5-bc9a-d79994d435d7\") " pod="openshift-network-operator/iptables-alerter-6kv6p" Sep 09 21:09:52.533567 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.532092 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/96490314-f8b7-439d-86e4-f61142aa887f-run\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.534138 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.534113 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f4e4311b-1130-4e07-8052-599cbb8ef158-agent-certs\") pod \"konnectivity-agent-bf4gk\" (UID: \"f4e4311b-1130-4e07-8052-599cbb8ef158\") " pod="kube-system/konnectivity-agent-bf4gk" Sep 09 21:09:52.534486 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.534458 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96490314-f8b7-439d-86e4-f61142aa887f-tmp\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.534594 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.534490 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/96490314-f8b7-439d-86e4-f61142aa887f-etc-tuned\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.540551 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.540524 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7r6gh\" (UniqueName: \"kubernetes.io/projected/436fa819-8e30-43d8-ae10-7c9d0cbc04c5-kube-api-access-7r6gh\") pod \"node-ca-srt4v\" (UID: \"436fa819-8e30-43d8-ae10-7c9d0cbc04c5\") " pod="openshift-image-registry/node-ca-srt4v" Sep 09 21:09:52.540749 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.540714 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jblhz\" (UniqueName: \"kubernetes.io/projected/96490314-f8b7-439d-86e4-f61142aa887f-kube-api-access-jblhz\") pod \"tuned-57dnk\" (UID: \"96490314-f8b7-439d-86e4-f61142aa887f\") " pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.541095 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.541068 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmwbk\" (UniqueName: \"kubernetes.io/projected/73b17a8d-a244-4ce5-bc9a-d79994d435d7-kube-api-access-xmwbk\") pod \"iptables-alerter-6kv6p\" (UID: \"73b17a8d-a244-4ce5-bc9a-d79994d435d7\") " pod="openshift-network-operator/iptables-alerter-6kv6p" Sep 09 21:09:52.542319 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.542295 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgl4s\" (UniqueName: \"kubernetes.io/projected/1abf2101-20e2-4e99-bb20-206110f86504-kube-api-access-kgl4s\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:09:52.613674 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.613572 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:09:52.624607 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.624579 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" Sep 09 21:09:52.637592 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.637559 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nxm74" Sep 09 21:09:52.642732 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.642701 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sz8l8" Sep 09 21:09:52.650881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.650848 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-j4r27" Sep 09 21:09:52.660920 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.660896 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6kv6p" Sep 09 21:09:52.670901 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.670875 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bf4gk" Sep 09 21:09:52.678852 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.678827 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-57dnk" Sep 09 21:09:52.682787 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:52.682770 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-srt4v" Sep 09 21:09:53.035800 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:53.035700 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl58d\" (UniqueName: \"kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d\") pod \"network-check-target-vk8h4\" (UID: \"7ed0ff95-e5da-49df-b563-a0988e9e8fe3\") " pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:09:53.035800 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:53.035763 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:09:53.036054 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:53.035825 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:53.036054 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:53.035893 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:09:53.036054 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:53.035912 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:09:53.036054 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:53.035916 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:09:53.036054 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:53.035933 2553 projected.go:196] Error preparing data for projected volume kube-api-access-wl58d for pod openshift-network-diagnostics/network-check-target-vk8h4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:09:53.036054 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:53.035935 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:09:53.036054 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:53.035981 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret podName:a77a1afc-9eda-42b4-857c-2967b6d149d8 nodeName:}" failed. No retries permitted until 2026-09-09 21:09:54.035961378 +0000 UTC m=+4.142705200 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret") pod "global-pull-secret-syncer-tbgn7" (UID: "a77a1afc-9eda-42b4-857c-2967b6d149d8") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:09:53.036054 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:53.036010 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs podName:1abf2101-20e2-4e99-bb20-206110f86504 nodeName:}" failed. No retries permitted until 2026-09-09 21:09:54.035987682 +0000 UTC m=+4.142731490 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs") pod "network-metrics-daemon-55b8r" (UID: "1abf2101-20e2-4e99-bb20-206110f86504") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:09:53.036054 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:53.036033 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d podName:7ed0ff95-e5da-49df-b563-a0988e9e8fe3 nodeName:}" failed. No retries permitted until 2026-09-09 21:09:54.036021711 +0000 UTC m=+4.142765517 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wl58d" (UniqueName: "kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d") pod "network-check-target-vk8h4" (UID: "7ed0ff95-e5da-49df-b563-a0988e9e8fe3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:09:53.366888 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:53.364555 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 21:04:52 +0000 UTC" deadline="2028-05-29 06:26:48.141046104 +0000 UTC" Sep 09 21:09:53.366888 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:53.364596 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15057h16m54.776454893s" Sep 09 21:09:53.512410 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:53.512163 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96490314_f8b7_439d_86e4_f61142aa887f.slice/crio-7d7ad16f35a543679d46823334fca823eaf17770f9c2bce3b709ac7fb56e914d WatchSource:0}: Error finding container 7d7ad16f35a543679d46823334fca823eaf17770f9c2bce3b709ac7fb56e914d: Status 404 returned error can't find the container with id 7d7ad16f35a543679d46823334fca823eaf17770f9c2bce3b709ac7fb56e914d Sep 09 21:09:53.515509 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:53.514335 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod436fa819_8e30_43d8_ae10_7c9d0cbc04c5.slice/crio-188b2e61c1951c7558b074f427e65da5979e687eb2eabe85b23067178583fbfe WatchSource:0}: Error finding container 188b2e61c1951c7558b074f427e65da5979e687eb2eabe85b23067178583fbfe: Status 404 returned error can't find the container with id 188b2e61c1951c7558b074f427e65da5979e687eb2eabe85b23067178583fbfe Sep 09 21:09:53.520534 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:53.520504 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91388845_f81d_42fa_b7ec_5e50730971ea.slice/crio-3e0aa5d0f39b79cec4d1d745db0159edcf219d64848931b98920daa689f19970 WatchSource:0}: Error finding container 3e0aa5d0f39b79cec4d1d745db0159edcf219d64848931b98920daa689f19970: Status 404 returned error can't find the container with id 3e0aa5d0f39b79cec4d1d745db0159edcf219d64848931b98920daa689f19970 Sep 09 21:09:53.521479 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:53.521458 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4e4311b_1130_4e07_8052_599cbb8ef158.slice/crio-0a2e58d0b71fe912d77a6a8febf82fb37e96ecf8307e7bf213c916c6ed4a1951 WatchSource:0}: Error finding container 0a2e58d0b71fe912d77a6a8febf82fb37e96ecf8307e7bf213c916c6ed4a1951: Status 404 returned error can't find the container with id 0a2e58d0b71fe912d77a6a8febf82fb37e96ecf8307e7bf213c916c6ed4a1951 Sep 09 21:09:53.522857 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:53.522587 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74a4c44e_98e5_4128_aee2_84987c8f66a4.slice/crio-2d8af9bef51b0275491cde410003681bb89d6ec6dd46511819e89376f8ab09e7 WatchSource:0}: Error finding container 2d8af9bef51b0275491cde410003681bb89d6ec6dd46511819e89376f8ab09e7: Status 404 returned error can't find the container with id 2d8af9bef51b0275491cde410003681bb89d6ec6dd46511819e89376f8ab09e7 Sep 09 21:09:53.522915 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:53.522889 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd70094ac_92b5_4cfd_bad5_522da65bbf9e.slice/crio-d14083b42dcedc4a06ba15733ed35f2523c08519c363a1113094d1190b5f6f4c WatchSource:0}: Error finding container d14083b42dcedc4a06ba15733ed35f2523c08519c363a1113094d1190b5f6f4c: Status 404 returned error can't find the container with id d14083b42dcedc4a06ba15733ed35f2523c08519c363a1113094d1190b5f6f4c Sep 09 21:09:53.524569 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:53.524272 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d6e1e2_96f1_4c46_b850_43586ded2151.slice/crio-4fdada5339b3bb98b429ffa5be931dd41355e97203d05eb0c214189a0eb412d4 WatchSource:0}: Error finding container 4fdada5339b3bb98b429ffa5be931dd41355e97203d05eb0c214189a0eb412d4: Status 404 returned error can't find the container with id 4fdada5339b3bb98b429ffa5be931dd41355e97203d05eb0c214189a0eb412d4 Sep 09 21:09:53.527253 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:09:53.527231 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b72de97_ed02_4e23_84db_3d6a80a3e685.slice/crio-a59eac14fc227501dd3650188d830641f0164b843237d89fe12111b1ccd594ab WatchSource:0}: Error finding container a59eac14fc227501dd3650188d830641f0164b843237d89fe12111b1ccd594ab: Status 404 returned error can't find the container with id a59eac14fc227501dd3650188d830641f0164b843237d89fe12111b1ccd594ab Sep 09 21:09:54.045454 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.044378 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:54.045454 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.044451 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl58d\" (UniqueName: \"kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d\") pod \"network-check-target-vk8h4\" (UID: \"7ed0ff95-e5da-49df-b563-a0988e9e8fe3\") " pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:09:54.045454 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.044486 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:09:54.045454 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:54.044638 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:09:54.045454 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:54.044704 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs podName:1abf2101-20e2-4e99-bb20-206110f86504 nodeName:}" failed. No retries permitted until 2026-09-09 21:09:56.044684397 +0000 UTC m=+6.151428218 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs") pod "network-metrics-daemon-55b8r" (UID: "1abf2101-20e2-4e99-bb20-206110f86504") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:09:54.045454 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:54.045131 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:09:54.045454 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:54.045195 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret podName:a77a1afc-9eda-42b4-857c-2967b6d149d8 nodeName:}" failed. No retries permitted until 2026-09-09 21:09:56.045165338 +0000 UTC m=+6.151909149 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret") pod "global-pull-secret-syncer-tbgn7" (UID: "a77a1afc-9eda-42b4-857c-2967b6d149d8") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:09:54.045454 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:54.045278 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:09:54.045454 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:54.045292 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:09:54.045454 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:54.045304 2553 projected.go:196] Error preparing data for projected volume kube-api-access-wl58d for pod openshift-network-diagnostics/network-check-target-vk8h4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:09:54.045454 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:54.045339 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d podName:7ed0ff95-e5da-49df-b563-a0988e9e8fe3 nodeName:}" failed. No retries permitted until 2026-09-09 21:09:56.045328407 +0000 UTC m=+6.152072216 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wl58d" (UniqueName: "kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d") pod "network-check-target-vk8h4" (UID: "7ed0ff95-e5da-49df-b563-a0988e9e8fe3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:09:54.365069 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.364933 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 21:04:52 +0000 UTC" deadline="2028-04-28 11:36:12.722575086 +0000 UTC" Sep 09 21:09:54.365069 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.365003 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14318h26m18.357576512s" Sep 09 21:09:54.386921 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.386054 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:09:54.386921 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:54.386221 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:09:54.386921 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.386626 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:09:54.386921 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:54.386708 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:09:54.386921 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.386780 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:54.386921 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:54.386853 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:09:54.406124 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.406080 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" event={"ID":"74a4c44e-98e5-4128-aee2-84987c8f66a4","Type":"ContainerStarted","Data":"2d8af9bef51b0275491cde410003681bb89d6ec6dd46511819e89376f8ab09e7"} Sep 09 21:09:54.414792 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.414684 2553 generic.go:356] "Generic (PLEG): container finished" podID="8edc6fae6fa3feede68fb3f030e50c39" containerID="2af1692574430caca8ebb8b21acc0f50e1a1379ef4ca4aa1ab9fdf7ef31fa6d2" exitCode=0 Sep 09 21:09:54.414978 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.414911 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" event={"ID":"8edc6fae6fa3feede68fb3f030e50c39","Type":"ContainerDied","Data":"2af1692574430caca8ebb8b21acc0f50e1a1379ef4ca4aa1ab9fdf7ef31fa6d2"} Sep 09 21:09:54.423408 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.423366 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6kv6p" event={"ID":"73b17a8d-a244-4ce5-bc9a-d79994d435d7","Type":"ContainerStarted","Data":"aa5a7a76de56c40160b2bacc076d8dfa8a2515078123ede06683f59eec7acbb5"} Sep 09 21:09:54.435010 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.434882 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j4r27" event={"ID":"e0d6e1e2-96f1-4c46-b850-43586ded2151","Type":"ContainerStarted","Data":"4fdada5339b3bb98b429ffa5be931dd41355e97203d05eb0c214189a0eb412d4"} Sep 09 21:09:54.456370 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.456213 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" event={"ID":"d70094ac-92b5-4cfd-bad5-522da65bbf9e","Type":"ContainerStarted","Data":"d14083b42dcedc4a06ba15733ed35f2523c08519c363a1113094d1190b5f6f4c"} Sep 09 21:09:54.472917 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.472819 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bf4gk" event={"ID":"f4e4311b-1130-4e07-8052-599cbb8ef158","Type":"ContainerStarted","Data":"0a2e58d0b71fe912d77a6a8febf82fb37e96ecf8307e7bf213c916c6ed4a1951"} Sep 09 21:09:54.488635 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.488577 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sz8l8" event={"ID":"91388845-f81d-42fa-b7ec-5e50730971ea","Type":"ContainerStarted","Data":"3e0aa5d0f39b79cec4d1d745db0159edcf219d64848931b98920daa689f19970"} Sep 09 21:09:54.529496 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.529391 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-srt4v" event={"ID":"436fa819-8e30-43d8-ae10-7c9d0cbc04c5","Type":"ContainerStarted","Data":"188b2e61c1951c7558b074f427e65da5979e687eb2eabe85b23067178583fbfe"} Sep 09 21:09:54.544470 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.544383 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-57dnk" event={"ID":"96490314-f8b7-439d-86e4-f61142aa887f","Type":"ContainerStarted","Data":"7d7ad16f35a543679d46823334fca823eaf17770f9c2bce3b709ac7fb56e914d"} Sep 09 21:09:54.575783 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.575731 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-115.ec2.internal" event={"ID":"0531880037de699c667aef422549784e","Type":"ContainerStarted","Data":"a4e5e365aa07e541dd3b9218ab0ccad0e01104c284c83885db7ed1d5b0bb88c0"} Sep 09 21:09:54.594000 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:54.593926 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxm74" event={"ID":"8b72de97-ed02-4e23-84db-3d6a80a3e685","Type":"ContainerStarted","Data":"a59eac14fc227501dd3650188d830641f0164b843237d89fe12111b1ccd594ab"} Sep 09 21:09:55.614028 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:55.613783 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" event={"ID":"8edc6fae6fa3feede68fb3f030e50c39","Type":"ContainerStarted","Data":"804c60426c92ec43c910ea9a54b32a25a65a274b79f5f32e2f08b7762a22c6d7"} Sep 09 21:09:55.636974 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:55.636907 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-115.ec2.internal" podStartSLOduration=4.636830105 podStartE2EDuration="4.636830105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:09:55.636415439 +0000 UTC m=+5.743159296" watchObservedRunningTime="2026-09-09 21:09:55.636830105 +0000 UTC m=+5.743573935" Sep 09 21:09:55.637872 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:55.637835 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-115.ec2.internal" podStartSLOduration=4.637819551 podStartE2EDuration="4.637819551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:09:54.5937455 +0000 UTC m=+4.700489331" watchObservedRunningTime="2026-09-09 21:09:55.637819551 +0000 UTC m=+5.744563382" Sep 09 21:09:56.062911 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:56.062792 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:09:56.062911 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:56.062888 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:56.063373 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:56.062929 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl58d\" (UniqueName: \"kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d\") pod \"network-check-target-vk8h4\" (UID: \"7ed0ff95-e5da-49df-b563-a0988e9e8fe3\") " pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:09:56.063373 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:56.063085 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:09:56.063373 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:56.063105 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:09:56.063373 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:56.063119 2553 projected.go:196] Error preparing data for projected volume kube-api-access-wl58d for pod openshift-network-diagnostics/network-check-target-vk8h4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:09:56.063373 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:56.063205 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d podName:7ed0ff95-e5da-49df-b563-a0988e9e8fe3 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:00.063184952 +0000 UTC m=+10.169928782 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wl58d" (UniqueName: "kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d") pod "network-check-target-vk8h4" (UID: "7ed0ff95-e5da-49df-b563-a0988e9e8fe3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:09:56.063373 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:56.063277 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:09:56.063373 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:56.063345 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret podName:a77a1afc-9eda-42b4-857c-2967b6d149d8 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:00.063326386 +0000 UTC m=+10.170070207 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret") pod "global-pull-secret-syncer-tbgn7" (UID: "a77a1afc-9eda-42b4-857c-2967b6d149d8") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:09:56.063373 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:56.063277 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:09:56.063871 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:56.063389 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs podName:1abf2101-20e2-4e99-bb20-206110f86504 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:00.063378289 +0000 UTC m=+10.170122097 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs") pod "network-metrics-daemon-55b8r" (UID: "1abf2101-20e2-4e99-bb20-206110f86504") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:09:56.384335 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:56.384214 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:09:56.384501 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:56.384356 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:09:56.384768 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:56.384750 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:09:56.384901 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:56.384881 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:09:56.385061 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:56.385046 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:56.385191 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:56.385145 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:09:58.383366 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:58.383335 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:09:58.383958 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:58.383442 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:09:58.383958 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:58.383813 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:09:58.383958 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:58.383890 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:09:58.384262 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:09:58.384244 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:09:58.384345 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:09:58.384324 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:00.101509 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:00.101469 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:00.101943 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:00.101525 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl58d\" (UniqueName: \"kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d\") pod \"network-check-target-vk8h4\" (UID: \"7ed0ff95-e5da-49df-b563-a0988e9e8fe3\") " pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:00.101943 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:00.101560 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:00.101943 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:00.101653 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:00.101943 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:00.101703 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:10:00.101943 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:00.101726 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:10:00.101943 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:00.101740 2553 projected.go:196] Error preparing data for projected volume kube-api-access-wl58d for pod openshift-network-diagnostics/network-check-target-vk8h4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:00.101943 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:00.101741 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret podName:a77a1afc-9eda-42b4-857c-2967b6d149d8 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:08.101720771 +0000 UTC m=+18.208464578 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret") pod "global-pull-secret-syncer-tbgn7" (UID: "a77a1afc-9eda-42b4-857c-2967b6d149d8") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:00.101943 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:00.101808 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d podName:7ed0ff95-e5da-49df-b563-a0988e9e8fe3 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:08.101790128 +0000 UTC m=+18.208533951 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wl58d" (UniqueName: "kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d") pod "network-check-target-vk8h4" (UID: "7ed0ff95-e5da-49df-b563-a0988e9e8fe3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:00.101943 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:00.101657 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:00.101943 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:00.101845 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs podName:1abf2101-20e2-4e99-bb20-206110f86504 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:08.101835387 +0000 UTC m=+18.208579195 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs") pod "network-metrics-daemon-55b8r" (UID: "1abf2101-20e2-4e99-bb20-206110f86504") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:00.387123 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:00.387011 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:00.387286 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:00.387133 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:00.387663 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:00.387426 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:00.387663 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:00.387531 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:00.387663 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:00.387564 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:00.387663 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:00.387660 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:02.384350 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:02.383596 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:02.384350 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:02.383728 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:02.384350 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:02.384076 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:02.384350 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:02.384176 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:02.384350 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:02.384241 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:02.384350 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:02.384313 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:04.383854 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:04.383820 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:04.384263 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:04.383859 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:04.384263 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:04.383825 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:04.384263 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:04.383955 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:04.384263 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:04.384016 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:04.384263 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:04.384122 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:06.383335 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:06.383295 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:06.383786 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:06.383295 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:06.383786 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:06.383425 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:06.383786 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:06.383295 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:06.383786 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:06.383547 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:06.383786 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:06.383580 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:08.160448 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:08.160353 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:08.160448 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:08.160405 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl58d\" (UniqueName: \"kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d\") pod \"network-check-target-vk8h4\" (UID: \"7ed0ff95-e5da-49df-b563-a0988e9e8fe3\") " pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:08.160448 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:08.160436 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:08.161022 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:08.160520 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:08.161022 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:08.160552 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:08.161022 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:08.160578 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:10:08.161022 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:08.160595 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:10:08.161022 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:08.160604 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret podName:a77a1afc-9eda-42b4-857c-2967b6d149d8 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:24.160588165 +0000 UTC m=+34.267331969 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret") pod "global-pull-secret-syncer-tbgn7" (UID: "a77a1afc-9eda-42b4-857c-2967b6d149d8") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:08.161022 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:08.160607 2553 projected.go:196] Error preparing data for projected volume kube-api-access-wl58d for pod openshift-network-diagnostics/network-check-target-vk8h4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:08.161022 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:08.160623 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs podName:1abf2101-20e2-4e99-bb20-206110f86504 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:24.160613049 +0000 UTC m=+34.267356855 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs") pod "network-metrics-daemon-55b8r" (UID: "1abf2101-20e2-4e99-bb20-206110f86504") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:08.161022 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:08.160637 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d podName:7ed0ff95-e5da-49df-b563-a0988e9e8fe3 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:24.160630301 +0000 UTC m=+34.267374113 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wl58d" (UniqueName: "kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d") pod "network-check-target-vk8h4" (UID: "7ed0ff95-e5da-49df-b563-a0988e9e8fe3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:08.383425 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:08.383381 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:08.383425 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:08.383416 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:08.383674 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:08.383497 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:08.383674 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:08.383518 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:08.383674 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:08.383580 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:08.383811 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:08.383715 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:10.384072 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:10.384037 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:10.384548 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:10.384159 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:10.384548 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:10.384186 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:10.384548 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:10.384230 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:10.384548 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:10.384303 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:10.384548 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:10.384380 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:12.383247 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.383074 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:12.383912 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.383074 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:12.383912 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:12.383324 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:12.383912 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:12.383437 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:12.383912 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.383074 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:12.383912 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:12.383528 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:12.663028 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.662948 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" event={"ID":"d70094ac-92b5-4cfd-bad5-522da65bbf9e","Type":"ContainerStarted","Data":"bd1d4264b2418d7268acf4f7eb9f6dfe85eeb078b0eb1d606021b77db9838573"} Sep 09 21:10:12.663028 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.663023 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" event={"ID":"d70094ac-92b5-4cfd-bad5-522da65bbf9e","Type":"ContainerStarted","Data":"00d8ee04a3a8320f948c5fd57179ea87ad343e55b90906375f867e9aea76943c"} Sep 09 21:10:12.663245 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.663070 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" event={"ID":"d70094ac-92b5-4cfd-bad5-522da65bbf9e","Type":"ContainerStarted","Data":"cba23273246b5f87b1c828a66bfe526929bb202c507f59dc998d282647fb3fc8"} Sep 09 21:10:12.663245 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.663114 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" event={"ID":"d70094ac-92b5-4cfd-bad5-522da65bbf9e","Type":"ContainerStarted","Data":"04d165eef34e81347038483318f228dae24a6f275810da5f51f4823b47afc648"} Sep 09 21:10:12.663245 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.663141 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" event={"ID":"d70094ac-92b5-4cfd-bad5-522da65bbf9e","Type":"ContainerStarted","Data":"562ca8a24e0bf2619cd63d140f6fe7192cd24a6cc3adc8f141b31d9af30f44a3"} Sep 09 21:10:12.663245 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.663178 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" event={"ID":"d70094ac-92b5-4cfd-bad5-522da65bbf9e","Type":"ContainerStarted","Data":"266d6639e1b61e38b104a916fd53941359ef30acb5e2c1d16fc93624c12f48d5"} Sep 09 21:10:12.664454 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.664432 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bf4gk" event={"ID":"f4e4311b-1130-4e07-8052-599cbb8ef158","Type":"ContainerStarted","Data":"e35e6bdf7c91149881cdc6f499ffd5f683ba3f97dc9de2ac6371033571c12c76"} Sep 09 21:10:12.665823 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.665791 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sz8l8" event={"ID":"91388845-f81d-42fa-b7ec-5e50730971ea","Type":"ContainerStarted","Data":"8ce299a6ddfe434a5c633919ca4d0021462b2234318efd4d4851e4f8930ff805"} Sep 09 21:10:12.667186 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.667165 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-srt4v" event={"ID":"436fa819-8e30-43d8-ae10-7c9d0cbc04c5","Type":"ContainerStarted","Data":"0864847a4f55e654c7af393b1520078d974c212cc9cddd75ee0b372895b65ccd"} Sep 09 21:10:12.668601 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.668571 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-57dnk" event={"ID":"96490314-f8b7-439d-86e4-f61142aa887f","Type":"ContainerStarted","Data":"fa7ef8462ad0c0a1247574f3ba20e8c3b687e4c61213a26531c7ae7ed1ec01c2"} Sep 09 21:10:12.669943 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.669925 2553 generic.go:356] "Generic (PLEG): container finished" podID="8b72de97-ed02-4e23-84db-3d6a80a3e685" containerID="4a351d308e41619fdf6a4775afedd725d46ab54611dbf946316ee41dd4f501b6" exitCode=0 Sep 09 21:10:12.670055 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.669990 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxm74" event={"ID":"8b72de97-ed02-4e23-84db-3d6a80a3e685","Type":"ContainerDied","Data":"4a351d308e41619fdf6a4775afedd725d46ab54611dbf946316ee41dd4f501b6"} Sep 09 21:10:12.671502 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.671480 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" event={"ID":"74a4c44e-98e5-4128-aee2-84987c8f66a4","Type":"ContainerStarted","Data":"84053a3e20cc41a96a1a3a23004065ac643c3275c77182b4ecb55842b5193153"} Sep 09 21:10:12.673046 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.673029 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-j4r27" event={"ID":"e0d6e1e2-96f1-4c46-b850-43586ded2151","Type":"ContainerStarted","Data":"5c070ad6877f235a58d382b776778b5846144fc9388d08f7dcaeb6bbdf204f8b"} Sep 09 21:10:12.680314 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.680276 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bf4gk" podStartSLOduration=7.301384893 podStartE2EDuration="21.680263133s" totalImagesPullingTime="14.37887824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:12.679648347 +0000 UTC m=+22.786392172" watchObservedRunningTime="2026-09-09 21:10:12.680263133 +0000 UTC m=+22.787006958" Sep 09 21:10:12.717436 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.717382 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-j4r27" podStartSLOduration=3.603856619 podStartE2EDuration="21.7173661s" totalImagesPullingTime="18.113509481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:12.717130382 +0000 UTC m=+22.823874210" watchObservedRunningTime="2026-09-09 21:10:12.7173661 +0000 UTC m=+22.824109926" Sep 09 21:10:12.749225 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.749178 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sz8l8" podStartSLOduration=3.692106232 podStartE2EDuration="21.749162654s" totalImagesPullingTime="18.057056422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:12.730887441 +0000 UTC m=+22.837631279" watchObservedRunningTime="2026-09-09 21:10:12.749162654 +0000 UTC m=+22.855906482" Sep 09 21:10:12.749555 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.749533 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-57dnk" podStartSLOduration=3.6727017440000003 podStartE2EDuration="21.749529408s" totalImagesPullingTime="18.076827664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:12.748974496 +0000 UTC m=+22.855718334" watchObservedRunningTime="2026-09-09 21:10:12.749529408 +0000 UTC m=+22.856273233" Sep 09 21:10:12.764443 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:12.764397 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-srt4v" podStartSLOduration=3.700616584 podStartE2EDuration="21.764383058s" totalImagesPullingTime="18.063766474s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:12.764144495 +0000 UTC m=+22.870888340" watchObservedRunningTime="2026-09-09 21:10:12.764383058 +0000 UTC m=+22.871126917" Sep 09 21:10:13.008937 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:13.008682 2553 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Sep 09 21:10:13.281345 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:13.281269 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bf4gk" Sep 09 21:10:13.281930 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:13.281905 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bf4gk" Sep 09 21:10:13.386232 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:13.386122 2553 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T21:10:13.008933459Z","UUID":"6815725e-425f-422b-82e5-12a6e216acc0","Handler":null,"Name":"","Endpoint":""} Sep 09 21:10:13.387533 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:13.387509 2553 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Sep 09 21:10:13.387533 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:13.387537 2553 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Sep 09 21:10:13.677637 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:13.677603 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" event={"ID":"74a4c44e-98e5-4128-aee2-84987c8f66a4","Type":"ContainerStarted","Data":"069d0aedf9e71e50ba43ffb875706f17a43d1faaeff334a247b19cdc0b3b7625"} Sep 09 21:10:13.679271 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:13.679231 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6kv6p" event={"ID":"73b17a8d-a244-4ce5-bc9a-d79994d435d7","Type":"ContainerStarted","Data":"4b253b4e946ea3f1cbada24f23a69c692df9ae180d274c2a0f2954b6f4415b56"} Sep 09 21:10:13.680243 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:13.680225 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bf4gk" Sep 09 21:10:13.680771 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:13.680753 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bf4gk" Sep 09 21:10:13.696725 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:13.696659 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6kv6p" podStartSLOduration=4.64416057 podStartE2EDuration="22.696639146s" totalImagesPullingTime="18.052478576s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:13.696263073 +0000 UTC m=+23.803006924" watchObservedRunningTime="2026-09-09 21:10:13.696639146 +0000 UTC m=+23.803382973" Sep 09 21:10:14.384043 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:14.384005 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:14.384233 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:14.384164 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:14.384233 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:14.384189 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:14.384353 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:14.384293 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:14.384464 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:14.384441 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:14.384574 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:14.384551 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:14.684188 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:14.684147 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" event={"ID":"74a4c44e-98e5-4128-aee2-84987c8f66a4","Type":"ContainerStarted","Data":"7f98d986c3e045deac726e362ce152f1e6283ec746d9e8aa13734036337ea4e7"} Sep 09 21:10:14.687913 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:14.687863 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" event={"ID":"d70094ac-92b5-4cfd-bad5-522da65bbf9e","Type":"ContainerStarted","Data":"742e74d9c862d560b02d768c8e47e9e62378a7167fd003674159a05e231d4782"} Sep 09 21:10:16.383742 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:16.383707 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:16.384324 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:16.383704 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:16.384324 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:16.383854 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:16.384324 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:16.383936 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:16.384324 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:16.383712 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:16.384324 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:16.384026 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:17.696787 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:17.696593 2553 generic.go:356] "Generic (PLEG): container finished" podID="8b72de97-ed02-4e23-84db-3d6a80a3e685" containerID="41686752ca09776c5c5e8e6dc455a3663275787aeff38e3c25385bfea6c3d0ec" exitCode=0 Sep 09 21:10:17.697529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:17.696647 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxm74" event={"ID":"8b72de97-ed02-4e23-84db-3d6a80a3e685","Type":"ContainerDied","Data":"41686752ca09776c5c5e8e6dc455a3663275787aeff38e3c25385bfea6c3d0ec"} Sep 09 21:10:17.700628 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:17.700601 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" event={"ID":"d70094ac-92b5-4cfd-bad5-522da65bbf9e","Type":"ContainerStarted","Data":"d5c373c37646c81f315429bba953a34aba9e834630b47a32aa01533ab6c3a55c"} Sep 09 21:10:17.700885 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:17.700867 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:10:17.700951 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:17.700930 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:10:17.715970 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:17.715943 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:10:17.716178 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:17.716045 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:10:17.726248 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:17.726201 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r46bf" podStartSLOduration=6.37763583 podStartE2EDuration="26.726185548s" totalImagesPullingTime="20.348549718s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:14.701580043 +0000 UTC m=+24.808323889" watchObservedRunningTime="2026-09-09 21:10:17.726185548 +0000 UTC m=+27.832929374" Sep 09 21:10:17.759405 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:17.759354 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" podStartSLOduration=9.644415949 podStartE2EDuration="27.759339253s" totalImagesPullingTime="18.114923304s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:17.759073797 +0000 UTC m=+27.865817633" watchObservedRunningTime="2026-09-09 21:10:17.759339253 +0000 UTC m=+27.866083079" Sep 09 21:10:18.383408 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:18.383369 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:18.383664 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:18.383369 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:18.383664 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:18.383505 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:18.383664 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:18.383369 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:18.383664 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:18.383556 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:18.383840 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:18.383688 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:18.703353 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:18.703195 2553 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 09 21:10:18.789068 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:18.789034 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tbgn7"] Sep 09 21:10:18.789261 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:18.789156 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:18.789261 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:18.789250 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:18.793787 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:18.793764 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-55b8r"] Sep 09 21:10:18.793912 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:18.793878 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:18.794021 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:18.793998 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:18.797416 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:18.797393 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vk8h4"] Sep 09 21:10:18.797527 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:18.797480 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:18.797575 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:18.797557 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:19.707587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:19.707555 2553 generic.go:356] "Generic (PLEG): container finished" podID="8b72de97-ed02-4e23-84db-3d6a80a3e685" containerID="0f3744f9ea0692590f0c9a4bf8310fdeb3ec9cd4e773b1f284c5fe66b7528716" exitCode=0 Sep 09 21:10:19.707957 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:19.707635 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxm74" event={"ID":"8b72de97-ed02-4e23-84db-3d6a80a3e685","Type":"ContainerDied","Data":"0f3744f9ea0692590f0c9a4bf8310fdeb3ec9cd4e773b1f284c5fe66b7528716"} Sep 09 21:10:19.707957 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:19.707691 2553 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Sep 09 21:10:20.383838 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:20.383805 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:20.384059 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:20.383846 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:20.384059 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:20.383924 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:20.384059 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:20.383983 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:20.384059 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:20.383996 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:20.384350 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:20.384074 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:20.711906 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:20.711869 2553 generic.go:356] "Generic (PLEG): container finished" podID="8b72de97-ed02-4e23-84db-3d6a80a3e685" containerID="0df610b63562b62085cffef0856cb808b27066a3f85657b2aac278e6cee907fb" exitCode=0 Sep 09 21:10:20.712278 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:20.711917 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxm74" event={"ID":"8b72de97-ed02-4e23-84db-3d6a80a3e685","Type":"ContainerDied","Data":"0df610b63562b62085cffef0856cb808b27066a3f85657b2aac278e6cee907fb"} Sep 09 21:10:20.791528 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:20.791497 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:10:22.384067 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:22.384030 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:22.384560 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:22.384030 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:22.384560 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:22.384169 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tbgn7" podUID="a77a1afc-9eda-42b4-857c-2967b6d149d8" Sep 09 21:10:22.384560 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:22.384244 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk8h4" podUID="7ed0ff95-e5da-49df-b563-a0988e9e8fe3" Sep 09 21:10:22.384560 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:22.384030 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:22.384560 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:22.384347 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-55b8r" podUID="1abf2101-20e2-4e99-bb20-206110f86504" Sep 09 21:10:24.177896 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.177848 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl58d\" (UniqueName: \"kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d\") pod \"network-check-target-vk8h4\" (UID: \"7ed0ff95-e5da-49df-b563-a0988e9e8fe3\") " pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:24.178384 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.177913 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:24.178384 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.177970 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:24.178384 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.178041 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Sep 09 21:10:24.178384 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.178074 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Sep 09 21:10:24.178384 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.178076 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:24.178384 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.178087 2553 projected.go:196] Error preparing data for projected volume kube-api-access-wl58d for pod openshift-network-diagnostics/network-check-target-vk8h4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:24.178384 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.178088 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:24.178384 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.178161 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret podName:a77a1afc-9eda-42b4-857c-2967b6d149d8 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:56.178141702 +0000 UTC m=+66.284885508 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret") pod "global-pull-secret-syncer-tbgn7" (UID: "a77a1afc-9eda-42b4-857c-2967b6d149d8") : object "kube-system"/"original-pull-secret" not registered Sep 09 21:10:24.178384 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.178180 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d podName:7ed0ff95-e5da-49df-b563-a0988e9e8fe3 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:56.178170868 +0000 UTC m=+66.284914675 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wl58d" (UniqueName: "kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d") pod "network-check-target-vk8h4" (UID: "7ed0ff95-e5da-49df-b563-a0988e9e8fe3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Sep 09 21:10:24.178384 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.178197 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs podName:1abf2101-20e2-4e99-bb20-206110f86504 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:56.178188614 +0000 UTC m=+66.284932423 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs") pod "network-metrics-daemon-55b8r" (UID: "1abf2101-20e2-4e99-bb20-206110f86504") : object "openshift-multus"/"metrics-daemon-secret" not registered Sep 09 21:10:24.223237 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.223207 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-115.ec2.internal" event="NodeReady" Sep 09 21:10:24.223410 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.223349 2553 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Sep 09 21:10:24.273059 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.273029 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-94b4b97c6-jlwgh"] Sep 09 21:10:24.350386 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.350349 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw"] Sep 09 21:10:24.350561 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.350489 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.353941 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.353915 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Sep 09 21:10:24.353941 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.353930 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Sep 09 21:10:24.354355 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.354225 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xkzkp\"" Sep 09 21:10:24.354355 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.354246 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Sep 09 21:10:24.361053 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.361033 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Sep 09 21:10:24.363744 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.363690 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-9gxmg"] Sep 09 21:10:24.363851 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.363817 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:24.367301 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.367282 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Sep 09 21:10:24.367636 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.367617 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bpjcw\"" Sep 09 21:10:24.367738 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.367623 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Sep 09 21:10:24.367827 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.367806 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:10:24.386151 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.386116 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n"] Sep 09 21:10:24.386341 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.386190 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:24.386341 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.386191 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.386341 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.386191 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:24.386504 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.386485 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:24.390332 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.390307 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 21:10:24.390492 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.390377 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Sep 09 21:10:24.390492 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.390378 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7j9l\"" Sep 09 21:10:24.390492 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.390406 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:10:24.390825 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.390804 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 21:10:24.390930 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.390855 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Sep 09 21:10:24.390930 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.390893 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Sep 09 21:10:24.391030 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.390950 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Sep 09 21:10:24.391030 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.390804 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-r7s7l\"" Sep 09 21:10:24.391513 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.391493 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ck66v\"" Sep 09 21:10:24.391610 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.391563 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Sep 09 21:10:24.399274 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.399241 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Sep 09 21:10:24.401954 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.401936 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:24.405175 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.405025 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Sep 09 21:10:24.405175 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.405168 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gtnjz\"" Sep 09 21:10:24.405381 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.405294 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Sep 09 21:10:24.405381 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.405338 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Sep 09 21:10:24.405381 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.405373 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Sep 09 21:10:24.406608 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.406581 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-64f7976768-h7vtq"] Sep 09 21:10:24.425932 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.425895 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-7vkst"] Sep 09 21:10:24.426177 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.426154 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.429147 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.429084 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Sep 09 21:10:24.429147 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.429120 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Sep 09 21:10:24.429305 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.429166 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Sep 09 21:10:24.429305 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.429123 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-n75x8\"" Sep 09 21:10:24.429453 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.429434 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Sep 09 21:10:24.429690 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.429671 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Sep 09 21:10:24.429769 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.429671 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Sep 09 21:10:24.442037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.442003 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt"] Sep 09 21:10:24.442210 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.442132 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.444975 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.444950 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Sep 09 21:10:24.445641 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.445620 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-9kvjc\"" Sep 09 21:10:24.445641 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.445674 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Sep 09 21:10:24.445913 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.445881 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Sep 09 21:10:24.445962 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.445933 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Sep 09 21:10:24.451057 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.450967 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Sep 09 21:10:24.458119 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.458074 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-9x9vj"] Sep 09 21:10:24.458270 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.458205 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" Sep 09 21:10:24.461058 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.461037 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Sep 09 21:10:24.461649 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.461628 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Sep 09 21:10:24.461838 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.461823 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6kr56\"" Sep 09 21:10:24.462421 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.462405 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:10:24.462512 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.462440 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Sep 09 21:10:24.472196 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.472173 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-r5sd7"] Sep 09 21:10:24.472305 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.472276 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-9x9vj" Sep 09 21:10:24.474747 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.474725 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-n2qrc\"" Sep 09 21:10:24.479248 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.479225 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdw9k\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-kube-api-access-pdw9k\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.479408 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.479385 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6cec47f9-1808-4396-9ee3-4fc7c5cf1586-trusted-ca\") pod \"console-operator-85b454874b-9gxmg\" (UID: \"6cec47f9-1808-4396-9ee3-4fc7c5cf1586\") " pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.479535 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.479520 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psnj5\" (UniqueName: \"kubernetes.io/projected/6cec47f9-1808-4396-9ee3-4fc7c5cf1586-kube-api-access-psnj5\") pod \"console-operator-85b454874b-9gxmg\" (UID: \"6cec47f9-1808-4396-9ee3-4fc7c5cf1586\") " pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.479681 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.479662 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:24.479752 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.479697 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lj75\" (UniqueName: \"kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75\") pod \"cluster-samples-operator-7cdcfcf76c-f6tbw\" (UID: \"24725293-ce93-4439-92ff-0310c29b8e96\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:24.479752 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.479722 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6cec47f9-1808-4396-9ee3-4fc7c5cf1586-config\") pod \"console-operator-85b454874b-9gxmg\" (UID: \"6cec47f9-1808-4396-9ee3-4fc7c5cf1586\") " pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.479846 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.479766 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6cec47f9-1808-4396-9ee3-4fc7c5cf1586-serving-cert\") pod \"console-operator-85b454874b-9gxmg\" (UID: \"6cec47f9-1808-4396-9ee3-4fc7c5cf1586\") " pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.479903 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.479881 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ece35fe8-abce-464f-a888-9798a5725a40-image-registry-private-configuration\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.479966 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.479936 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4cfb52c0-2216-418e-adf2-80859d555aa9-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:24.480026 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.479973 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-f6tbw\" (UID: \"24725293-ce93-4439-92ff-0310c29b8e96\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:24.480026 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.479998 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.480026 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.480018 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ece35fe8-abce-464f-a888-9798a5725a40-trusted-ca\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.480200 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.480043 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-bound-sa-token\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.480200 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.480062 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4bcr\" (UniqueName: \"kubernetes.io/projected/4cfb52c0-2216-418e-adf2-80859d555aa9-kube-api-access-c4bcr\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:24.480200 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.480119 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ece35fe8-abce-464f-a888-9798a5725a40-ca-trust-extracted\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.480200 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.480145 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ece35fe8-abce-464f-a888-9798a5725a40-registry-certificates\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.480200 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.480169 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ece35fe8-abce-464f-a888-9798a5725a40-installation-pull-secrets\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.490354 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.490317 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf"] Sep 09 21:10:24.490354 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.490343 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-r5sd7" Sep 09 21:10:24.492932 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.492910 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-q7fwt\"" Sep 09 21:10:24.493552 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.493531 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:10:24.493645 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.493556 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Sep 09 21:10:24.498029 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.498000 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:24.512685 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.512652 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-94b4b97c6-jlwgh"] Sep 09 21:10:24.512856 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.512697 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ncrcz"] Sep 09 21:10:24.512856 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.512771 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" Sep 09 21:10:24.515763 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.515738 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Sep 09 21:10:24.515921 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.515891 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-n842d\"" Sep 09 21:10:24.516211 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.516178 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Sep 09 21:10:24.516845 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.516826 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Sep 09 21:10:24.516954 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.516900 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Sep 09 21:10:24.522559 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.522539 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qxszf"] Sep 09 21:10:24.522683 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.522668 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ncrcz" Sep 09 21:10:24.525651 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.525627 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v6fqv\"" Sep 09 21:10:24.525754 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.525652 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Sep 09 21:10:24.525848 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.525832 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Sep 09 21:10:24.525943 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.525929 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Sep 09 21:10:24.537121 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.537079 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw"] Sep 09 21:10:24.537258 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.537142 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-9gxmg"] Sep 09 21:10:24.537258 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.537166 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt"] Sep 09 21:10:24.537258 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.537185 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-9x9vj"] Sep 09 21:10:24.537258 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.537206 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qxszf"] Sep 09 21:10:24.537258 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.537229 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-r5sd7"] Sep 09 21:10:24.537258 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.537249 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n"] Sep 09 21:10:24.537520 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.537267 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64f7976768-h7vtq"] Sep 09 21:10:24.537520 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.537285 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-7vkst"] Sep 09 21:10:24.537520 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.537297 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:24.537520 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.537302 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ncrcz"] Sep 09 21:10:24.537520 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.537434 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf"] Sep 09 21:10:24.539996 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.539977 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Sep 09 21:10:24.540313 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.540291 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Sep 09 21:10:24.540522 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.540507 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7flhb\"" Sep 09 21:10:24.581363 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581330 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-f6tbw\" (UID: \"24725293-ce93-4439-92ff-0310c29b8e96\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:24.581553 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581366 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.581553 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581409 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ece35fe8-abce-464f-a888-9798a5725a40-trusted-ca\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.581553 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581426 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-bound-sa-token\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.581553 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581447 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ece35fe8-abce-464f-a888-9798a5725a40-installation-pull-secrets\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.581553 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581464 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ece35fe8-abce-464f-a888-9798a5725a40-ca-trust-extracted\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.581553 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581484 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjc7k\" (UniqueName: \"kubernetes.io/projected/c8e97526-a820-45e2-9696-618ff30b6688-kube-api-access-kjc7k\") pod \"ingress-canary-ncrcz\" (UID: \"c8e97526-a820-45e2-9696-618ff30b6688\") " pod="openshift-ingress-canary/ingress-canary-ncrcz" Sep 09 21:10:24.581553 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581500 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trhzw\" (UniqueName: \"kubernetes.io/projected/0a512f34-bdcf-44ee-a4bd-2c2b49cb7553-kube-api-access-trhzw\") pod \"service-ca-operator-859f94c4b-hghqt\" (UID: \"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" Sep 09 21:10:24.581553 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581517 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-default-certificate\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.581553 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581538 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581563 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-stats-auth\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581588 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjjxl\" (UniqueName: \"kubernetes.io/projected/097d572f-d83a-493b-a181-59385bb2afc1-kube-api-access-qjjxl\") pod \"volume-data-source-validator-695c895bd6-r5sd7\" (UID: \"097d572f-d83a-493b-a181-59385bb2afc1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-r5sd7" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581610 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6cec47f9-1808-4396-9ee3-4fc7c5cf1586-trusted-ca\") pod \"console-operator-85b454874b-9gxmg\" (UID: \"6cec47f9-1808-4396-9ee3-4fc7c5cf1586\") " pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581635 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lj75\" (UniqueName: \"kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75\") pod \"cluster-samples-operator-7cdcfcf76c-f6tbw\" (UID: \"24725293-ce93-4439-92ff-0310c29b8e96\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581654 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581671 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6cec47f9-1808-4396-9ee3-4fc7c5cf1586-config\") pod \"console-operator-85b454874b-9gxmg\" (UID: \"6cec47f9-1808-4396-9ee3-4fc7c5cf1586\") " pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581689 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6cec47f9-1808-4396-9ee3-4fc7c5cf1586-serving-cert\") pod \"console-operator-85b454874b-9gxmg\" (UID: \"6cec47f9-1808-4396-9ee3-4fc7c5cf1586\") " pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581721 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4cfb52c0-2216-418e-adf2-80859d555aa9-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581739 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0896f595-88a0-4e6e-956a-6abd1cbc0f97-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-gstjf\" (UID: \"0896f595-88a0-4e6e-956a-6abd1cbc0f97\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581764 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb25b5c9-2b1a-4286-9668-a57b10e163d6-serving-cert\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581781 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb25b5c9-2b1a-4286-9668-a57b10e163d6-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581796 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpb2n\" (UniqueName: \"kubernetes.io/projected/bb25b5c9-2b1a-4286-9668-a57b10e163d6-kube-api-access-vpb2n\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581820 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4bcr\" (UniqueName: \"kubernetes.io/projected/4cfb52c0-2216-418e-adf2-80859d555aa9-kube-api-access-c4bcr\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581844 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g559s\" (UniqueName: \"kubernetes.io/projected/05b70482-3a69-427f-8ce3-922c764c470b-kube-api-access-g559s\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.581904 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581866 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5vxq\" (UniqueName: \"kubernetes.io/projected/0896f595-88a0-4e6e-956a-6abd1cbc0f97-kube-api-access-n5vxq\") pod \"kube-storage-version-migrator-operator-8d7dd4675-gstjf\" (UID: \"0896f595-88a0-4e6e-956a-6abd1cbc0f97\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" Sep 09 21:10:24.582451 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581883 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w46x6\" (UniqueName: \"kubernetes.io/projected/6576a5ea-032b-4155-9deb-bae78b644d73-kube-api-access-w46x6\") pod \"network-check-source-685456655f-9x9vj\" (UID: \"6576a5ea-032b-4155-9deb-bae78b644d73\") " pod="openshift-network-diagnostics/network-check-source-685456655f-9x9vj" Sep 09 21:10:24.582451 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581910 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0896f595-88a0-4e6e-956a-6abd1cbc0f97-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-gstjf\" (UID: \"0896f595-88a0-4e6e-956a-6abd1cbc0f97\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" Sep 09 21:10:24.582451 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.581938 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ece35fe8-abce-464f-a888-9798a5725a40-registry-certificates\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.582451 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.582113 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 21:10:24.582451 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.582193 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:25.082171172 +0000 UTC m=+35.188914995 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : secret "samples-operator-tls" not found Sep 09 21:10:24.582451 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.582119 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:10:24.582451 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.582339 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-94b4b97c6-jlwgh: secret "image-registry-tls" not found Sep 09 21:10:24.582451 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.582376 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls podName:ece35fe8-abce-464f-a888-9798a5725a40 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:25.08236481 +0000 UTC m=+35.189108614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls") pod "image-registry-94b4b97c6-jlwgh" (UID: "ece35fe8-abce-464f-a888-9798a5725a40") : secret "image-registry-tls" not found Sep 09 21:10:24.582451 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.582423 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdw9k\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-kube-api-access-pdw9k\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.582451 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.582451 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8e97526-a820-45e2-9696-618ff30b6688-cert\") pod \"ingress-canary-ncrcz\" (UID: \"c8e97526-a820-45e2-9696-618ff30b6688\") " pod="openshift-ingress-canary/ingress-canary-ncrcz" Sep 09 21:10:24.582761 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.582475 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a512f34-bdcf-44ee-a4bd-2c2b49cb7553-config\") pod \"service-ca-operator-859f94c4b-hghqt\" (UID: \"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" Sep 09 21:10:24.582761 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.582517 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psnj5\" (UniqueName: \"kubernetes.io/projected/6cec47f9-1808-4396-9ee3-4fc7c5cf1586-kube-api-access-psnj5\") pod \"console-operator-85b454874b-9gxmg\" (UID: \"6cec47f9-1808-4396-9ee3-4fc7c5cf1586\") " pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.582761 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.582629 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ece35fe8-abce-464f-a888-9798a5725a40-registry-certificates\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.582761 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.582689 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bb25b5c9-2b1a-4286-9668-a57b10e163d6-snapshots\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.582761 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.582714 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/6cec47f9-1808-4396-9ee3-4fc7c5cf1586-config\") pod \"console-operator-85b454874b-9gxmg\" (UID: \"6cec47f9-1808-4396-9ee3-4fc7c5cf1586\") " pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.582969 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.582787 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/4cfb52c0-2216-418e-adf2-80859d555aa9-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:24.582969 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.582836 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.582969 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.582928 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ece35fe8-abce-464f-a888-9798a5725a40-image-registry-private-configuration\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.582969 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.582959 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 21:10:24.583142 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.582971 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb25b5c9-2b1a-4286-9668-a57b10e163d6-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.583142 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.583023 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls podName:4cfb52c0-2216-418e-adf2-80859d555aa9 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:25.083006778 +0000 UTC m=+35.189750601 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-6pg2n" (UID: "4cfb52c0-2216-418e-adf2-80859d555aa9") : secret "cluster-monitoring-operator-tls" not found Sep 09 21:10:24.583142 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.583066 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a512f34-bdcf-44ee-a4bd-2c2b49cb7553-serving-cert\") pod \"service-ca-operator-859f94c4b-hghqt\" (UID: \"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" Sep 09 21:10:24.583263 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.583147 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb25b5c9-2b1a-4286-9668-a57b10e163d6-tmp\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.583912 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.583867 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ece35fe8-abce-464f-a888-9798a5725a40-ca-trust-extracted\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.583912 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.583892 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ece35fe8-abce-464f-a888-9798a5725a40-trusted-ca\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.584052 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.583958 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6cec47f9-1808-4396-9ee3-4fc7c5cf1586-trusted-ca\") pod \"console-operator-85b454874b-9gxmg\" (UID: \"6cec47f9-1808-4396-9ee3-4fc7c5cf1586\") " pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.587093 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.586933 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/6cec47f9-1808-4396-9ee3-4fc7c5cf1586-serving-cert\") pod \"console-operator-85b454874b-9gxmg\" (UID: \"6cec47f9-1808-4396-9ee3-4fc7c5cf1586\") " pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.587210 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.586971 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ece35fe8-abce-464f-a888-9798a5725a40-image-registry-private-configuration\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.587348 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.587314 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ece35fe8-abce-464f-a888-9798a5725a40-installation-pull-secrets\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.596557 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.596501 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-bound-sa-token\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.598169 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.598142 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lj75\" (UniqueName: \"kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75\") pod \"cluster-samples-operator-7cdcfcf76c-f6tbw\" (UID: \"24725293-ce93-4439-92ff-0310c29b8e96\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:24.598547 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.598501 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdw9k\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-kube-api-access-pdw9k\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:24.598736 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.598715 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psnj5\" (UniqueName: \"kubernetes.io/projected/6cec47f9-1808-4396-9ee3-4fc7c5cf1586-kube-api-access-psnj5\") pod \"console-operator-85b454874b-9gxmg\" (UID: \"6cec47f9-1808-4396-9ee3-4fc7c5cf1586\") " pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.599062 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.599036 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4bcr\" (UniqueName: \"kubernetes.io/projected/4cfb52c0-2216-418e-adf2-80859d555aa9-kube-api-access-c4bcr\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:24.615484 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.615410 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:24.683810 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.683771 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8e97526-a820-45e2-9696-618ff30b6688-cert\") pod \"ingress-canary-ncrcz\" (UID: \"c8e97526-a820-45e2-9696-618ff30b6688\") " pod="openshift-ingress-canary/ingress-canary-ncrcz" Sep 09 21:10:24.683810 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.683811 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a512f34-bdcf-44ee-a4bd-2c2b49cb7553-config\") pod \"service-ca-operator-859f94c4b-hghqt\" (UID: \"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" Sep 09 21:10:24.683810 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.683829 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bb25b5c9-2b1a-4286-9668-a57b10e163d6-snapshots\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.684136 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.683850 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:24.684136 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.683953 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.684136 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.683983 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb25b5c9-2b1a-4286-9668-a57b10e163d6-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.684136 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684000 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a512f34-bdcf-44ee-a4bd-2c2b49cb7553-serving-cert\") pod \"service-ca-operator-859f94c4b-hghqt\" (UID: \"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" Sep 09 21:10:24.684136 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684025 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-tmp-dir\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:24.684136 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684051 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb25b5c9-2b1a-4286-9668-a57b10e163d6-tmp\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.684136 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684074 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-config-volume\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:24.684487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684164 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjc7k\" (UniqueName: \"kubernetes.io/projected/c8e97526-a820-45e2-9696-618ff30b6688-kube-api-access-kjc7k\") pod \"ingress-canary-ncrcz\" (UID: \"c8e97526-a820-45e2-9696-618ff30b6688\") " pod="openshift-ingress-canary/ingress-canary-ncrcz" Sep 09 21:10:24.684487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684191 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trhzw\" (UniqueName: \"kubernetes.io/projected/0a512f34-bdcf-44ee-a4bd-2c2b49cb7553-kube-api-access-trhzw\") pod \"service-ca-operator-859f94c4b-hghqt\" (UID: \"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" Sep 09 21:10:24.684487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684229 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-default-certificate\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.684487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684257 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.684487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684279 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-stats-auth\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.684487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684302 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjjxl\" (UniqueName: \"kubernetes.io/projected/097d572f-d83a-493b-a181-59385bb2afc1-kube-api-access-qjjxl\") pod \"volume-data-source-validator-695c895bd6-r5sd7\" (UID: \"097d572f-d83a-493b-a181-59385bb2afc1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-r5sd7" Sep 09 21:10:24.684487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684356 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cl8v\" (UniqueName: \"kubernetes.io/projected/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-kube-api-access-9cl8v\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:24.684487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684407 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0896f595-88a0-4e6e-956a-6abd1cbc0f97-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-gstjf\" (UID: \"0896f595-88a0-4e6e-956a-6abd1cbc0f97\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" Sep 09 21:10:24.684487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684434 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb25b5c9-2b1a-4286-9668-a57b10e163d6-serving-cert\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.684487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684459 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb25b5c9-2b1a-4286-9668-a57b10e163d6-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.684487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684483 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0a512f34-bdcf-44ee-a4bd-2c2b49cb7553-config\") pod \"service-ca-operator-859f94c4b-hghqt\" (UID: \"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" Sep 09 21:10:24.685025 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684492 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpb2n\" (UniqueName: \"kubernetes.io/projected/bb25b5c9-2b1a-4286-9668-a57b10e163d6-kube-api-access-vpb2n\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.685025 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684544 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bb25b5c9-2b1a-4286-9668-a57b10e163d6-snapshots\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.685025 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684559 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g559s\" (UniqueName: \"kubernetes.io/projected/05b70482-3a69-427f-8ce3-922c764c470b-kube-api-access-g559s\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.685025 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684593 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5vxq\" (UniqueName: \"kubernetes.io/projected/0896f595-88a0-4e6e-956a-6abd1cbc0f97-kube-api-access-n5vxq\") pod \"kube-storage-version-migrator-operator-8d7dd4675-gstjf\" (UID: \"0896f595-88a0-4e6e-956a-6abd1cbc0f97\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" Sep 09 21:10:24.685025 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684622 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w46x6\" (UniqueName: \"kubernetes.io/projected/6576a5ea-032b-4155-9deb-bae78b644d73-kube-api-access-w46x6\") pod \"network-check-source-685456655f-9x9vj\" (UID: \"6576a5ea-032b-4155-9deb-bae78b644d73\") " pod="openshift-network-diagnostics/network-check-source-685456655f-9x9vj" Sep 09 21:10:24.685025 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.684655 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0896f595-88a0-4e6e-956a-6abd1cbc0f97-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-gstjf\" (UID: \"0896f595-88a0-4e6e-956a-6abd1cbc0f97\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" Sep 09 21:10:24.685349 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.685189 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0896f595-88a0-4e6e-956a-6abd1cbc0f97-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-gstjf\" (UID: \"0896f595-88a0-4e6e-956a-6abd1cbc0f97\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" Sep 09 21:10:24.685412 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.685368 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bb25b5c9-2b1a-4286-9668-a57b10e163d6-tmp\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.685494 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.685478 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 21:10:24.685550 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.685541 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs podName:05b70482-3a69-427f-8ce3-922c764c470b nodeName:}" failed. No retries permitted until 2026-09-09 21:10:25.185523016 +0000 UTC m=+35.292266842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs") pod "router-default-64f7976768-h7vtq" (UID: "05b70482-3a69-427f-8ce3-922c764c470b") : secret "router-metrics-certs-default" not found Sep 09 21:10:24.685616 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.685571 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb25b5c9-2b1a-4286-9668-a57b10e163d6-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.685880 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.685857 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle podName:05b70482-3a69-427f-8ce3-922c764c470b nodeName:}" failed. No retries permitted until 2026-09-09 21:10:25.185836907 +0000 UTC m=+35.292580731 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle") pod "router-default-64f7976768-h7vtq" (UID: "05b70482-3a69-427f-8ce3-922c764c470b") : configmap references non-existent config key: service-ca.crt Sep 09 21:10:24.685993 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.685974 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bb25b5c9-2b1a-4286-9668-a57b10e163d6-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.687186 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.687158 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0a512f34-bdcf-44ee-a4bd-2c2b49cb7553-serving-cert\") pod \"service-ca-operator-859f94c4b-hghqt\" (UID: \"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" Sep 09 21:10:24.687306 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.687222 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bb25b5c9-2b1a-4286-9668-a57b10e163d6-serving-cert\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.687306 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.687261 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c8e97526-a820-45e2-9696-618ff30b6688-cert\") pod \"ingress-canary-ncrcz\" (UID: \"c8e97526-a820-45e2-9696-618ff30b6688\") " pod="openshift-ingress-canary/ingress-canary-ncrcz" Sep 09 21:10:24.687496 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.687476 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-stats-auth\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.687685 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.687671 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0896f595-88a0-4e6e-956a-6abd1cbc0f97-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-gstjf\" (UID: \"0896f595-88a0-4e6e-956a-6abd1cbc0f97\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" Sep 09 21:10:24.688581 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.688558 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-default-certificate\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.694433 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.694409 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjc7k\" (UniqueName: \"kubernetes.io/projected/c8e97526-a820-45e2-9696-618ff30b6688-kube-api-access-kjc7k\") pod \"ingress-canary-ncrcz\" (UID: \"c8e97526-a820-45e2-9696-618ff30b6688\") " pod="openshift-ingress-canary/ingress-canary-ncrcz" Sep 09 21:10:24.694541 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.694413 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g559s\" (UniqueName: \"kubernetes.io/projected/05b70482-3a69-427f-8ce3-922c764c470b-kube-api-access-g559s\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:24.698549 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.698515 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w46x6\" (UniqueName: \"kubernetes.io/projected/6576a5ea-032b-4155-9deb-bae78b644d73-kube-api-access-w46x6\") pod \"network-check-source-685456655f-9x9vj\" (UID: \"6576a5ea-032b-4155-9deb-bae78b644d73\") " pod="openshift-network-diagnostics/network-check-source-685456655f-9x9vj" Sep 09 21:10:24.699178 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.699163 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trhzw\" (UniqueName: \"kubernetes.io/projected/0a512f34-bdcf-44ee-a4bd-2c2b49cb7553-kube-api-access-trhzw\") pod \"service-ca-operator-859f94c4b-hghqt\" (UID: \"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" Sep 09 21:10:24.699457 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.699433 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpb2n\" (UniqueName: \"kubernetes.io/projected/bb25b5c9-2b1a-4286-9668-a57b10e163d6-kube-api-access-vpb2n\") pod \"insights-operator-76bcd6ccf5-7vkst\" (UID: \"bb25b5c9-2b1a-4286-9668-a57b10e163d6\") " pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.699636 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.699619 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5vxq\" (UniqueName: \"kubernetes.io/projected/0896f595-88a0-4e6e-956a-6abd1cbc0f97-kube-api-access-n5vxq\") pod \"kube-storage-version-migrator-operator-8d7dd4675-gstjf\" (UID: \"0896f595-88a0-4e6e-956a-6abd1cbc0f97\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" Sep 09 21:10:24.699779 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.699762 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjjxl\" (UniqueName: \"kubernetes.io/projected/097d572f-d83a-493b-a181-59385bb2afc1-kube-api-access-qjjxl\") pod \"volume-data-source-validator-695c895bd6-r5sd7\" (UID: \"097d572f-d83a-493b-a181-59385bb2afc1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-r5sd7" Sep 09 21:10:24.708933 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.708913 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:24.754745 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.754709 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" Sep 09 21:10:24.768399 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.768360 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" Sep 09 21:10:24.782934 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.782914 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-9x9vj" Sep 09 21:10:24.785293 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.785268 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:24.785402 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.785312 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-tmp-dir\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:24.785402 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.785332 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-config-volume\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:24.785509 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.785404 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cl8v\" (UniqueName: \"kubernetes.io/projected/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-kube-api-access-9cl8v\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:24.785509 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.785452 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:10:24.785604 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:24.785535 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls podName:ca9ee75e-2df9-47f5-b556-977f41f7eeb1 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:25.285511676 +0000 UTC m=+35.392255488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls") pod "dns-default-qxszf" (UID: "ca9ee75e-2df9-47f5-b556-977f41f7eeb1") : secret "dns-default-metrics-tls" not found Sep 09 21:10:24.785656 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.785596 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-tmp-dir\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:24.785916 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.785896 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-config-volume\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:24.796281 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.796253 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cl8v\" (UniqueName: \"kubernetes.io/projected/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-kube-api-access-9cl8v\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:24.801181 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.801154 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-r5sd7" Sep 09 21:10:24.825057 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.824970 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" Sep 09 21:10:24.832431 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:24.832333 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ncrcz" Sep 09 21:10:25.029779 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.029754 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-9gxmg"] Sep 09 21:10:25.052312 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.052289 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-7vkst"] Sep 09 21:10:25.078078 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.078053 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt"] Sep 09 21:10:25.079876 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:25.079851 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cec47f9_1808_4396_9ee3_4fc7c5cf1586.slice/crio-d003a044f8244782b240d22f855b0adea92de54bc3cfcd3ba9d9455f92dfe3e2 WatchSource:0}: Error finding container d003a044f8244782b240d22f855b0adea92de54bc3cfcd3ba9d9455f92dfe3e2: Status 404 returned error can't find the container with id d003a044f8244782b240d22f855b0adea92de54bc3cfcd3ba9d9455f92dfe3e2 Sep 09 21:10:25.080319 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:25.080295 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb25b5c9_2b1a_4286_9668_a57b10e163d6.slice/crio-de5ca6a1e7587a07b2a6a9575bc7d6277c8c43eba809320587c9b1319da85dae WatchSource:0}: Error finding container de5ca6a1e7587a07b2a6a9575bc7d6277c8c43eba809320587c9b1319da85dae: Status 404 returned error can't find the container with id de5ca6a1e7587a07b2a6a9575bc7d6277c8c43eba809320587c9b1319da85dae Sep 09 21:10:25.081397 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:25.081377 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a512f34_bdcf_44ee_a4bd_2c2b49cb7553.slice/crio-3c8216263747392dc0a1d431fb3e69d18cc288d54e115e69c45bcfd6d72b0299 WatchSource:0}: Error finding container 3c8216263747392dc0a1d431fb3e69d18cc288d54e115e69c45bcfd6d72b0299: Status 404 returned error can't find the container with id 3c8216263747392dc0a1d431fb3e69d18cc288d54e115e69c45bcfd6d72b0299 Sep 09 21:10:25.089227 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.089206 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-f6tbw\" (UID: \"24725293-ce93-4439-92ff-0310c29b8e96\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:25.089313 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.089240 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:25.089395 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:25.089339 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 21:10:25.089395 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:25.089391 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:26.089376551 +0000 UTC m=+36.196120356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : secret "samples-operator-tls" not found Sep 09 21:10:25.089514 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:25.089396 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:10:25.089514 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:25.089409 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-94b4b97c6-jlwgh: secret "image-registry-tls" not found Sep 09 21:10:25.089514 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:25.089458 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls podName:ece35fe8-abce-464f-a888-9798a5725a40 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:26.089443621 +0000 UTC m=+36.196187426 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls") pod "image-registry-94b4b97c6-jlwgh" (UID: "ece35fe8-abce-464f-a888-9798a5725a40") : secret "image-registry-tls" not found Sep 09 21:10:25.089514 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:25.089467 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 21:10:25.089514 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:25.089510 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls podName:4cfb52c0-2216-418e-adf2-80859d555aa9 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:26.08949914 +0000 UTC m=+36.196242950 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-6pg2n" (UID: "4cfb52c0-2216-418e-adf2-80859d555aa9") : secret "cluster-monitoring-operator-tls" not found Sep 09 21:10:25.089765 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.089314 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:25.115822 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.115774 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-r5sd7"] Sep 09 21:10:25.117835 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:25.117810 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod097d572f_d83a_493b_a181_59385bb2afc1.slice/crio-7b9da878e66c2e95f7892baadf3e9576a36f50e7f84d129cf2bcc9da26da74de WatchSource:0}: Error finding container 7b9da878e66c2e95f7892baadf3e9576a36f50e7f84d129cf2bcc9da26da74de: Status 404 returned error can't find the container with id 7b9da878e66c2e95f7892baadf3e9576a36f50e7f84d129cf2bcc9da26da74de Sep 09 21:10:25.123192 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.123167 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-9x9vj"] Sep 09 21:10:25.125513 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:25.125492 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6576a5ea_032b_4155_9deb_bae78b644d73.slice/crio-a89869060ca1e9e59c48bb5053a5f27831070d9eeba2467558854a657704bd39 WatchSource:0}: Error finding container a89869060ca1e9e59c48bb5053a5f27831070d9eeba2467558854a657704bd39: Status 404 returned error can't find the container with id a89869060ca1e9e59c48bb5053a5f27831070d9eeba2467558854a657704bd39 Sep 09 21:10:25.144058 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.144036 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf"] Sep 09 21:10:25.146049 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:25.146023 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0896f595_88a0_4e6e_956a_6abd1cbc0f97.slice/crio-21ce3fa14d0c2f47a83b4f9b9966cc50e7aa987f6697ddd1c9cfc323576010f1 WatchSource:0}: Error finding container 21ce3fa14d0c2f47a83b4f9b9966cc50e7aa987f6697ddd1c9cfc323576010f1: Status 404 returned error can't find the container with id 21ce3fa14d0c2f47a83b4f9b9966cc50e7aa987f6697ddd1c9cfc323576010f1 Sep 09 21:10:25.173797 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.173764 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ncrcz"] Sep 09 21:10:25.177202 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:25.177151 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8e97526_a820_45e2_9696_618ff30b6688.slice/crio-9c7cfa7aa468b6c048ceb8f3a5bfea6541e6c15c02f793465f7f21fe2d544f16 WatchSource:0}: Error finding container 9c7cfa7aa468b6c048ceb8f3a5bfea6541e6c15c02f793465f7f21fe2d544f16: Status 404 returned error can't find the container with id 9c7cfa7aa468b6c048ceb8f3a5bfea6541e6c15c02f793465f7f21fe2d544f16 Sep 09 21:10:25.191206 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.191177 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:25.191959 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.191337 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:25.191959 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:25.191502 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 21:10:25.191959 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:25.191560 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs podName:05b70482-3a69-427f-8ce3-922c764c470b nodeName:}" failed. No retries permitted until 2026-09-09 21:10:26.191540791 +0000 UTC m=+36.298284603 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs") pod "router-default-64f7976768-h7vtq" (UID: "05b70482-3a69-427f-8ce3-922c764c470b") : secret "router-metrics-certs-default" not found Sep 09 21:10:25.191959 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:25.191638 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle podName:05b70482-3a69-427f-8ce3-922c764c470b nodeName:}" failed. No retries permitted until 2026-09-09 21:10:26.191625313 +0000 UTC m=+36.298369130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle") pod "router-default-64f7976768-h7vtq" (UID: "05b70482-3a69-427f-8ce3-922c764c470b") : configmap references non-existent config key: service-ca.crt Sep 09 21:10:25.221006 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.220964 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:25.292770 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.292730 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:25.292958 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:25.292931 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:10:25.293037 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:25.293024 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls podName:ca9ee75e-2df9-47f5-b556-977f41f7eeb1 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:26.293001634 +0000 UTC m=+36.399745450 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls") pod "dns-default-qxszf" (UID: "ca9ee75e-2df9-47f5-b556-977f41f7eeb1") : secret "dns-default-metrics-tls" not found Sep 09 21:10:25.725843 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.725799 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" event={"ID":"0896f595-88a0-4e6e-956a-6abd1cbc0f97","Type":"ContainerStarted","Data":"21ce3fa14d0c2f47a83b4f9b9966cc50e7aa987f6697ddd1c9cfc323576010f1"} Sep 09 21:10:25.727589 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.727551 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" event={"ID":"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553","Type":"ContainerStarted","Data":"3c8216263747392dc0a1d431fb3e69d18cc288d54e115e69c45bcfd6d72b0299"} Sep 09 21:10:25.734355 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.733388 2553 generic.go:356] "Generic (PLEG): container finished" podID="8b72de97-ed02-4e23-84db-3d6a80a3e685" containerID="632ef54d7dbe0a2dc5875b1273ac49948bc9caa1f76752f46bfabb7f3821501d" exitCode=0 Sep 09 21:10:25.734355 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.733467 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxm74" event={"ID":"8b72de97-ed02-4e23-84db-3d6a80a3e685","Type":"ContainerDied","Data":"632ef54d7dbe0a2dc5875b1273ac49948bc9caa1f76752f46bfabb7f3821501d"} Sep 09 21:10:25.741147 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.741048 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ncrcz" event={"ID":"c8e97526-a820-45e2-9696-618ff30b6688","Type":"ContainerStarted","Data":"9c7cfa7aa468b6c048ceb8f3a5bfea6541e6c15c02f793465f7f21fe2d544f16"} Sep 09 21:10:25.742861 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.742803 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-9x9vj" event={"ID":"6576a5ea-032b-4155-9deb-bae78b644d73","Type":"ContainerStarted","Data":"a89869060ca1e9e59c48bb5053a5f27831070d9eeba2467558854a657704bd39"} Sep 09 21:10:25.744965 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.744940 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" event={"ID":"bb25b5c9-2b1a-4286-9668-a57b10e163d6","Type":"ContainerStarted","Data":"de5ca6a1e7587a07b2a6a9575bc7d6277c8c43eba809320587c9b1319da85dae"} Sep 09 21:10:25.746371 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.746309 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" event={"ID":"6cec47f9-1808-4396-9ee3-4fc7c5cf1586","Type":"ContainerStarted","Data":"d003a044f8244782b240d22f855b0adea92de54bc3cfcd3ba9d9455f92dfe3e2"} Sep 09 21:10:25.748116 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:25.748080 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-r5sd7" event={"ID":"097d572f-d83a-493b-a181-59385bb2afc1","Type":"ContainerStarted","Data":"7b9da878e66c2e95f7892baadf3e9576a36f50e7f84d129cf2bcc9da26da74de"} Sep 09 21:10:26.102038 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.101943 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-f6tbw\" (UID: \"24725293-ce93-4439-92ff-0310c29b8e96\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:26.102216 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.102060 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:26.102216 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.102154 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:26.103040 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:26.102405 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 21:10:26.103040 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:26.102473 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls podName:4cfb52c0-2216-418e-adf2-80859d555aa9 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:28.102454486 +0000 UTC m=+38.209198296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-6pg2n" (UID: "4cfb52c0-2216-418e-adf2-80859d555aa9") : secret "cluster-monitoring-operator-tls" not found Sep 09 21:10:26.103040 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:26.102857 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 21:10:26.103040 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:26.102904 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:28.102888881 +0000 UTC m=+38.209632690 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : secret "samples-operator-tls" not found Sep 09 21:10:26.103040 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:26.102969 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:10:26.103040 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:26.102979 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-94b4b97c6-jlwgh: secret "image-registry-tls" not found Sep 09 21:10:26.103040 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:26.103008 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls podName:ece35fe8-abce-464f-a888-9798a5725a40 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:28.102998458 +0000 UTC m=+38.209742264 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls") pod "image-registry-94b4b97c6-jlwgh" (UID: "ece35fe8-abce-464f-a888-9798a5725a40") : secret "image-registry-tls" not found Sep 09 21:10:26.203690 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.203082 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:26.203690 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.203193 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:26.203690 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:26.203229 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 21:10:26.203690 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:26.203303 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs podName:05b70482-3a69-427f-8ce3-922c764c470b nodeName:}" failed. No retries permitted until 2026-09-09 21:10:28.20328215 +0000 UTC m=+38.310025959 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs") pod "router-default-64f7976768-h7vtq" (UID: "05b70482-3a69-427f-8ce3-922c764c470b") : secret "router-metrics-certs-default" not found Sep 09 21:10:26.203690 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:26.203372 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle podName:05b70482-3a69-427f-8ce3-922c764c470b nodeName:}" failed. No retries permitted until 2026-09-09 21:10:28.203355236 +0000 UTC m=+38.310099045 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle") pod "router-default-64f7976768-h7vtq" (UID: "05b70482-3a69-427f-8ce3-922c764c470b") : configmap references non-existent config key: service-ca.crt Sep 09 21:10:26.304686 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.304646 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:26.304920 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:26.304806 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:10:26.304981 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:26.304933 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls podName:ca9ee75e-2df9-47f5-b556-977f41f7eeb1 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:28.304913874 +0000 UTC m=+38.411657679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls") pod "dns-default-qxszf" (UID: "ca9ee75e-2df9-47f5-b556-977f41f7eeb1") : secret "dns-default-metrics-tls" not found Sep 09 21:10:26.769490 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.768864 2553 generic.go:356] "Generic (PLEG): container finished" podID="8b72de97-ed02-4e23-84db-3d6a80a3e685" containerID="bffc8ad24bde3d32787d421644f2e18ad2ded5b7e80912cef1630343b669ad3d" exitCode=0 Sep 09 21:10:26.769490 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.768930 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxm74" event={"ID":"8b72de97-ed02-4e23-84db-3d6a80a3e685","Type":"ContainerDied","Data":"bffc8ad24bde3d32787d421644f2e18ad2ded5b7e80912cef1630343b669ad3d"} Sep 09 21:10:26.834607 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.834569 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-9hd77"] Sep 09 21:10:26.854883 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.854851 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-9hd77"] Sep 09 21:10:26.855057 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.854989 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:26.858392 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.857802 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Sep 09 21:10:26.858392 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.857811 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zm5n7\"" Sep 09 21:10:26.858392 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.858119 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Sep 09 21:10:26.970072 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:26.970027 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:27.016430 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:27.015533 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7bd30f8a-b08d-444d-b317-a32e7e0152b3-nginx-conf\") pod \"networking-console-plugin-79977f9844-9hd77\" (UID: \"7bd30f8a-b08d-444d-b317-a32e7e0152b3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:27.016430 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:27.015607 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-9hd77\" (UID: \"7bd30f8a-b08d-444d-b317-a32e7e0152b3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:27.078361 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:27.078240 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:27.117333 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:27.116760 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7bd30f8a-b08d-444d-b317-a32e7e0152b3-nginx-conf\") pod \"networking-console-plugin-79977f9844-9hd77\" (UID: \"7bd30f8a-b08d-444d-b317-a32e7e0152b3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:27.117333 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:27.116831 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-9hd77\" (UID: \"7bd30f8a-b08d-444d-b317-a32e7e0152b3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:27.117333 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:27.116995 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 21:10:27.117333 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:27.117060 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert podName:7bd30f8a-b08d-444d-b317-a32e7e0152b3 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:27.617040027 +0000 UTC m=+37.723783849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-9hd77" (UID: "7bd30f8a-b08d-444d-b317-a32e7e0152b3") : secret "networking-console-plugin-cert" not found Sep 09 21:10:27.117993 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:27.117970 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/7bd30f8a-b08d-444d-b317-a32e7e0152b3-nginx-conf\") pod \"networking-console-plugin-79977f9844-9hd77\" (UID: \"7bd30f8a-b08d-444d-b317-a32e7e0152b3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:27.150328 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:27.150291 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:27.620723 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:27.620674 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-9hd77\" (UID: \"7bd30f8a-b08d-444d-b317-a32e7e0152b3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:27.621242 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:27.620942 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 21:10:27.621242 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:27.621006 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert podName:7bd30f8a-b08d-444d-b317-a32e7e0152b3 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:28.62098817 +0000 UTC m=+38.727731979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-9hd77" (UID: "7bd30f8a-b08d-444d-b317-a32e7e0152b3") : secret "networking-console-plugin-cert" not found Sep 09 21:10:28.127018 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:28.126973 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-f6tbw\" (UID: \"24725293-ce93-4439-92ff-0310c29b8e96\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:28.127018 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:28.127025 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:28.127304 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:28.127094 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:28.127304 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.127150 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 21:10:28.127304 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.127209 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 21:10:28.127304 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.127216 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:10:28.127304 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.127214 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:32.12719628 +0000 UTC m=+42.233940107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : secret "samples-operator-tls" not found Sep 09 21:10:28.127304 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.127237 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-94b4b97c6-jlwgh: secret "image-registry-tls" not found Sep 09 21:10:28.127304 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.127253 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls podName:4cfb52c0-2216-418e-adf2-80859d555aa9 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:32.127242976 +0000 UTC m=+42.233986781 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-6pg2n" (UID: "4cfb52c0-2216-418e-adf2-80859d555aa9") : secret "cluster-monitoring-operator-tls" not found Sep 09 21:10:28.127304 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.127301 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls podName:ece35fe8-abce-464f-a888-9798a5725a40 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:32.127284034 +0000 UTC m=+42.234027857 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls") pod "image-registry-94b4b97c6-jlwgh" (UID: "ece35fe8-abce-464f-a888-9798a5725a40") : secret "image-registry-tls" not found Sep 09 21:10:28.228376 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:28.228340 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:28.228551 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:28.228435 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:28.228605 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.228583 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 21:10:28.228642 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.228600 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle podName:05b70482-3a69-427f-8ce3-922c764c470b nodeName:}" failed. No retries permitted until 2026-09-09 21:10:32.228574187 +0000 UTC m=+42.335318012 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle") pod "router-default-64f7976768-h7vtq" (UID: "05b70482-3a69-427f-8ce3-922c764c470b") : configmap references non-existent config key: service-ca.crt Sep 09 21:10:28.228642 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.228634 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs podName:05b70482-3a69-427f-8ce3-922c764c470b nodeName:}" failed. No retries permitted until 2026-09-09 21:10:32.228622242 +0000 UTC m=+42.335366047 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs") pod "router-default-64f7976768-h7vtq" (UID: "05b70482-3a69-427f-8ce3-922c764c470b") : secret "router-metrics-certs-default" not found Sep 09 21:10:28.328860 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:28.328822 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:28.329030 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.328979 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:10:28.329125 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.329052 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls podName:ca9ee75e-2df9-47f5-b556-977f41f7eeb1 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:32.329033755 +0000 UTC m=+42.435777563 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls") pod "dns-default-qxszf" (UID: "ca9ee75e-2df9-47f5-b556-977f41f7eeb1") : secret "dns-default-metrics-tls" not found Sep 09 21:10:28.632428 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:28.632381 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-9hd77\" (UID: \"7bd30f8a-b08d-444d-b317-a32e7e0152b3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:28.632901 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.632575 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 21:10:28.632901 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:28.632657 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert podName:7bd30f8a-b08d-444d-b317-a32e7e0152b3 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:30.632636052 +0000 UTC m=+40.739379864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-9hd77" (UID: "7bd30f8a-b08d-444d-b317-a32e7e0152b3") : secret "networking-console-plugin-cert" not found Sep 09 21:10:29.864656 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:29.864624 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:29.965659 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:29.965618 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:30.650952 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:30.650911 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-9hd77\" (UID: \"7bd30f8a-b08d-444d-b317-a32e7e0152b3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:30.651180 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:30.651084 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 21:10:30.651251 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:30.651195 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert podName:7bd30f8a-b08d-444d-b317-a32e7e0152b3 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:34.651174663 +0000 UTC m=+44.757918483 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-9hd77" (UID: "7bd30f8a-b08d-444d-b317-a32e7e0152b3") : secret "networking-console-plugin-cert" not found Sep 09 21:10:32.167372 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:32.167325 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-f6tbw\" (UID: \"24725293-ce93-4439-92ff-0310c29b8e96\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:32.167947 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:32.167382 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:32.167947 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:32.167450 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:32.167947 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:32.167505 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 21:10:32.167947 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:32.167558 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:10:32.167947 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:32.167577 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-94b4b97c6-jlwgh: secret "image-registry-tls" not found Sep 09 21:10:32.167947 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:32.167591 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:40.167571038 +0000 UTC m=+50.274314843 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : secret "samples-operator-tls" not found Sep 09 21:10:32.167947 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:32.167640 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 21:10:32.167947 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:32.167673 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls podName:ece35fe8-abce-464f-a888-9798a5725a40 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:40.16765914 +0000 UTC m=+50.274402948 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls") pod "image-registry-94b4b97c6-jlwgh" (UID: "ece35fe8-abce-464f-a888-9798a5725a40") : secret "image-registry-tls" not found Sep 09 21:10:32.167947 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:32.167694 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls podName:4cfb52c0-2216-418e-adf2-80859d555aa9 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:40.167679884 +0000 UTC m=+50.274423687 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-6pg2n" (UID: "4cfb52c0-2216-418e-adf2-80859d555aa9") : secret "cluster-monitoring-operator-tls" not found Sep 09 21:10:32.268059 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:32.268023 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:32.268241 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:32.268139 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:32.268241 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:32.268210 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle podName:05b70482-3a69-427f-8ce3-922c764c470b nodeName:}" failed. No retries permitted until 2026-09-09 21:10:40.268192415 +0000 UTC m=+50.374936223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle") pod "router-default-64f7976768-h7vtq" (UID: "05b70482-3a69-427f-8ce3-922c764c470b") : configmap references non-existent config key: service-ca.crt Sep 09 21:10:32.268319 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:32.268256 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 21:10:32.268319 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:32.268294 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs podName:05b70482-3a69-427f-8ce3-922c764c470b nodeName:}" failed. No retries permitted until 2026-09-09 21:10:40.268283268 +0000 UTC m=+50.375027072 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs") pod "router-default-64f7976768-h7vtq" (UID: "05b70482-3a69-427f-8ce3-922c764c470b") : secret "router-metrics-certs-default" not found Sep 09 21:10:32.369528 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:32.369488 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:32.369697 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:32.369650 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:10:32.369737 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:32.369717 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls podName:ca9ee75e-2df9-47f5-b556-977f41f7eeb1 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:40.36970344 +0000 UTC m=+50.476447257 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls") pod "dns-default-qxszf" (UID: "ca9ee75e-2df9-47f5-b556-977f41f7eeb1") : secret "dns-default-metrics-tls" not found Sep 09 21:10:33.793621 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.793579 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" event={"ID":"0896f595-88a0-4e6e-956a-6abd1cbc0f97","Type":"ContainerStarted","Data":"f738b5163a0150e21a22d1b1695b59eafcdab7cfc43e5a2e7c0f6c0989be5ec0"} Sep 09 21:10:33.795578 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.795410 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" event={"ID":"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553","Type":"ContainerStarted","Data":"16900cdcf88bec1d829b1084c3e31b73b387619c0bcd989f192142bc05f8506d"} Sep 09 21:10:33.802834 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.802802 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nxm74" event={"ID":"8b72de97-ed02-4e23-84db-3d6a80a3e685","Type":"ContainerStarted","Data":"4a49951c4cf21900ff84a0bbb651782be890795e2367ce376ebb78a40900482f"} Sep 09 21:10:33.805052 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.805029 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ncrcz" event={"ID":"c8e97526-a820-45e2-9696-618ff30b6688","Type":"ContainerStarted","Data":"e25beaa4d05a98ba018b0c269d90cd48ba8d0050e2dc26b57cd23341f1cdaa25"} Sep 09 21:10:33.806756 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.806733 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-9x9vj" event={"ID":"6576a5ea-032b-4155-9deb-bae78b644d73","Type":"ContainerStarted","Data":"e12492916e5e36b8bf12aba7f97568b939173a427debe0b457bfeb3f34e9c35b"} Sep 09 21:10:33.808694 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.808673 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" event={"ID":"bb25b5c9-2b1a-4286-9668-a57b10e163d6","Type":"ContainerStarted","Data":"8d9396e09ab6f7c278e36482171d176f7799e45fa6d541bff162bc012a547047"} Sep 09 21:10:33.810821 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.810804 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/0.log" Sep 09 21:10:33.810924 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.810838 2553 generic.go:356] "Generic (PLEG): container finished" podID="6cec47f9-1808-4396-9ee3-4fc7c5cf1586" containerID="a841cba50e8b556ce23b47a3f2b84cd0e53d48efb8c6f65482796e15001f931e" exitCode=255 Sep 09 21:10:33.810924 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.810899 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" event={"ID":"6cec47f9-1808-4396-9ee3-4fc7c5cf1586","Type":"ContainerDied","Data":"a841cba50e8b556ce23b47a3f2b84cd0e53d48efb8c6f65482796e15001f931e"} Sep 09 21:10:33.811088 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.811071 2553 scope.go:117] "RemoveContainer" containerID="a841cba50e8b556ce23b47a3f2b84cd0e53d48efb8c6f65482796e15001f931e" Sep 09 21:10:33.812679 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.812657 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-r5sd7" event={"ID":"097d572f-d83a-493b-a181-59385bb2afc1","Type":"ContainerStarted","Data":"60c3ecbecb8acefcbcfce335b80e1ed3c58d73f8eb948776e43f8fa1206e45a0"} Sep 09 21:10:33.817732 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.817545 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" podStartSLOduration=5.17203087 podStartE2EDuration="12.817529157s" totalImagesPullingTime="7.645498287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:33.816455414 +0000 UTC m=+43.923199285" watchObservedRunningTime="2026-09-09 21:10:33.817529157 +0000 UTC m=+43.924272974" Sep 09 21:10:33.854073 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.853771 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ncrcz" podStartSLOduration=2.238784152 podStartE2EDuration="9.853736651s" totalImagesPullingTime="7.614952499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:33.849826704 +0000 UTC m=+43.956570549" watchObservedRunningTime="2026-09-09 21:10:33.853736651 +0000 UTC m=+43.960480478" Sep 09 21:10:33.923792 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.923732 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-r5sd7" podStartSLOduration=6.290224182 podStartE2EDuration="13.923710681s" totalImagesPullingTime="7.633486499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:33.879397983 +0000 UTC m=+43.986141809" watchObservedRunningTime="2026-09-09 21:10:33.923710681 +0000 UTC m=+44.030454509" Sep 09 21:10:33.924611 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.924577 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nxm74" podStartSLOduration=14.332344315 podStartE2EDuration="42.924568251s" totalImagesPullingTime="28.592223936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:33.922223491 +0000 UTC m=+44.028967348" watchObservedRunningTime="2026-09-09 21:10:33.924568251 +0000 UTC m=+44.031312080" Sep 09 21:10:33.970840 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:33.970781 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" podStartSLOduration=3.278873509 podStartE2EDuration="10.970760818s" totalImagesPullingTime="7.691887309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:33.969861936 +0000 UTC m=+44.076605779" watchObservedRunningTime="2026-09-09 21:10:33.970760818 +0000 UTC m=+44.077504645" Sep 09 21:10:34.000912 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.000858 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-9x9vj" podStartSLOduration=3.322705399 podStartE2EDuration="11.000837597s" totalImagesPullingTime="7.678132198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:33.990741248 +0000 UTC m=+44.097485090" watchObservedRunningTime="2026-09-09 21:10:34.000837597 +0000 UTC m=+44.107581426" Sep 09 21:10:34.014345 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.014247 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" podStartSLOduration=6.322286176 podStartE2EDuration="14.01422504s" totalImagesPullingTime="7.691938864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:34.012582322 +0000 UTC m=+44.119326175" watchObservedRunningTime="2026-09-09 21:10:34.01422504 +0000 UTC m=+44.120968868" Sep 09 21:10:34.029681 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.029627 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:34.123391 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.123355 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:34.201832 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.201797 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:34.310815 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.310731 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4ckvp"] Sep 09 21:10:34.335757 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.335723 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4ckvp"] Sep 09 21:10:34.335922 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.335874 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.338871 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.338836 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wl9jm\"" Sep 09 21:10:34.339112 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.339087 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Sep 09 21:10:34.339200 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.339125 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Sep 09 21:10:34.349664 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.349634 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:34.387174 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.387133 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.387356 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.387265 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6cmn\" (UniqueName: \"kubernetes.io/projected/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-api-access-p6cmn\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.387356 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.387295 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.387472 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.387356 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e908db9-3103-49d9-a462-b3bc3757fd89-crio-socket\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.387472 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.387441 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e908db9-3103-49d9-a462-b3bc3757fd89-data-volume\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.488082 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.488041 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6cmn\" (UniqueName: \"kubernetes.io/projected/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-api-access-p6cmn\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.488278 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.488092 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.488278 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.488192 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e908db9-3103-49d9-a462-b3bc3757fd89-crio-socket\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.488278 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.488247 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e908db9-3103-49d9-a462-b3bc3757fd89-data-volume\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.488415 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:34.488284 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:10:34.488415 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:34.488359 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls podName:9e908db9-3103-49d9-a462-b3bc3757fd89 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:34.988338453 +0000 UTC m=+45.095082263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4ckvp" (UID: "9e908db9-3103-49d9-a462-b3bc3757fd89") : secret "insights-runtime-extractor-tls" not found Sep 09 21:10:34.488535 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.488440 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e908db9-3103-49d9-a462-b3bc3757fd89-crio-socket\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.488535 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.488513 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.488626 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.488602 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e908db9-3103-49d9-a462-b3bc3757fd89-data-volume\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.488679 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:34.488636 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:10:34.488739 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:34.488696 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm podName:9e908db9-3103-49d9-a462-b3bc3757fd89 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:34.988679006 +0000 UTC m=+45.095422814 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4ckvp" (UID: "9e908db9-3103-49d9-a462-b3bc3757fd89") : configmap "kube-rbac-proxy" not found Sep 09 21:10:34.500554 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.500527 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6cmn\" (UniqueName: \"kubernetes.io/projected/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-api-access-p6cmn\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.690141 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.690086 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-9hd77\" (UID: \"7bd30f8a-b08d-444d-b317-a32e7e0152b3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:34.690308 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:34.690236 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 21:10:34.690308 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:34.690303 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert podName:7bd30f8a-b08d-444d-b317-a32e7e0152b3 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:42.690285605 +0000 UTC m=+52.797029408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-9hd77" (UID: "7bd30f8a-b08d-444d-b317-a32e7e0152b3") : secret "networking-console-plugin-cert" not found Sep 09 21:10:34.699263 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.699230 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:34.709146 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.709117 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:34.709292 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.709179 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:34.797963 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.797928 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:34.818115 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.818069 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/1.log" Sep 09 21:10:34.818559 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.818543 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/0.log" Sep 09 21:10:34.818627 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.818577 2553 generic.go:356] "Generic (PLEG): container finished" podID="6cec47f9-1808-4396-9ee3-4fc7c5cf1586" containerID="52f1acf97426c26f62fd41fc0a8a098bb6158b86c5bde453a8297ec2c3d8cf36" exitCode=255 Sep 09 21:10:34.818627 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.818611 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" event={"ID":"6cec47f9-1808-4396-9ee3-4fc7c5cf1586","Type":"ContainerDied","Data":"52f1acf97426c26f62fd41fc0a8a098bb6158b86c5bde453a8297ec2c3d8cf36"} Sep 09 21:10:34.828911 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.828877 2553 scope.go:117] "RemoveContainer" containerID="a841cba50e8b556ce23b47a3f2b84cd0e53d48efb8c6f65482796e15001f931e" Sep 09 21:10:34.829089 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.829004 2553 scope.go:117] "RemoveContainer" containerID="52f1acf97426c26f62fd41fc0a8a098bb6158b86c5bde453a8297ec2c3d8cf36" Sep 09 21:10:34.829394 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:34.829365 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-9gxmg_openshift-console-operator(6cec47f9-1808-4396-9ee3-4fc7c5cf1586)\"" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" podUID="6cec47f9-1808-4396-9ee3-4fc7c5cf1586" Sep 09 21:10:34.993804 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.993707 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.993952 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:34.993839 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:10:34.993952 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:34.993919 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm podName:9e908db9-3103-49d9-a462-b3bc3757fd89 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:35.993901754 +0000 UTC m=+46.100645559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4ckvp" (UID: "9e908db9-3103-49d9-a462-b3bc3757fd89") : configmap "kube-rbac-proxy" not found Sep 09 21:10:34.994022 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:34.993951 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:34.994126 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:34.994113 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:10:34.994176 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:34.994167 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls podName:9e908db9-3103-49d9-a462-b3bc3757fd89 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:35.994156836 +0000 UTC m=+46.100900645 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4ckvp" (UID: "9e908db9-3103-49d9-a462-b3bc3757fd89") : secret "insights-runtime-extractor-tls" not found Sep 09 21:10:35.025889 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:35.025859 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:35.117036 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:35.116996 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:35.179448 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:35.179410 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:35.419737 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:35.419653 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:35.600802 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:35.600766 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:35.823790 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:35.823764 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/1.log" Sep 09 21:10:35.824299 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:35.824191 2553 scope.go:117] "RemoveContainer" containerID="52f1acf97426c26f62fd41fc0a8a098bb6158b86c5bde453a8297ec2c3d8cf36" Sep 09 21:10:35.824396 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:35.824375 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-9gxmg_openshift-console-operator(6cec47f9-1808-4396-9ee3-4fc7c5cf1586)\"" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" podUID="6cec47f9-1808-4396-9ee3-4fc7c5cf1586" Sep 09 21:10:35.882082 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:35.882054 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sz8l8_91388845-f81d-42fa-b7ec-5e50730971ea/dns-node-resolver/0.log" Sep 09 21:10:36.016883 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:36.016844 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:36.017089 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:36.016947 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:36.017089 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:36.016965 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:10:36.017089 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:36.017025 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm podName:9e908db9-3103-49d9-a462-b3bc3757fd89 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:38.017010142 +0000 UTC m=+48.123753946 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4ckvp" (UID: "9e908db9-3103-49d9-a462-b3bc3757fd89") : configmap "kube-rbac-proxy" not found Sep 09 21:10:36.017089 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:36.017058 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:10:36.020744 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:36.017145 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls podName:9e908db9-3103-49d9-a462-b3bc3757fd89 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:38.017128642 +0000 UTC m=+48.123872461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4ckvp" (UID: "9e908db9-3103-49d9-a462-b3bc3757fd89") : secret "insights-runtime-extractor-tls" not found Sep 09 21:10:36.326859 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:36.326822 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:36.427677 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:36.427640 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:36.873089 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:36.873057 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:36.970115 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:36.970070 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:37.034200 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:37.034165 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:37.082979 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:37.082951 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-srt4v_436fa819-8e30-43d8-ae10-7c9d0cbc04c5/node-ca/0.log" Sep 09 21:10:37.682723 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:37.682691 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ncrcz_c8e97526-a820-45e2-9696-618ff30b6688/serve-healthcheck-canary/0.log" Sep 09 21:10:37.901065 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:37.901030 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" limit=1024 Sep 09 21:10:38.035536 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:38.035452 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:38.035687 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:38.035554 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:38.035687 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:38.035580 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Sep 09 21:10:38.035687 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:38.035653 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm podName:9e908db9-3103-49d9-a462-b3bc3757fd89 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:42.035634501 +0000 UTC m=+52.142378324 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm") pod "insights-runtime-extractor-4ckvp" (UID: "9e908db9-3103-49d9-a462-b3bc3757fd89") : configmap "kube-rbac-proxy" not found Sep 09 21:10:38.035687 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:38.035661 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:10:38.035818 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:38.035698 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls podName:9e908db9-3103-49d9-a462-b3bc3757fd89 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:42.035685992 +0000 UTC m=+52.142429798 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4ckvp" (UID: "9e908db9-3103-49d9-a462-b3bc3757fd89") : secret "insights-runtime-extractor-tls" not found Sep 09 21:10:40.254077 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:40.254031 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-f6tbw\" (UID: \"24725293-ce93-4439-92ff-0310c29b8e96\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:40.254077 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:40.254080 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:40.254563 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:40.254161 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:40.254563 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:40.254213 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Sep 09 21:10:40.254563 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:40.254266 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Sep 09 21:10:40.254563 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:40.254296 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-94b4b97c6-jlwgh: secret "image-registry-tls" not found Sep 09 21:10:40.254563 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:40.254283 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:56.254264208 +0000 UTC m=+66.361008030 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : secret "samples-operator-tls" not found Sep 09 21:10:40.254563 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:40.254343 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Sep 09 21:10:40.254563 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:40.254400 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls podName:4cfb52c0-2216-418e-adf2-80859d555aa9 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:56.254387435 +0000 UTC m=+66.361131291 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-6pg2n" (UID: "4cfb52c0-2216-418e-adf2-80859d555aa9") : secret "cluster-monitoring-operator-tls" not found Sep 09 21:10:40.254563 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:40.254414 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls podName:ece35fe8-abce-464f-a888-9798a5725a40 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:56.254407555 +0000 UTC m=+66.361151362 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls") pod "image-registry-94b4b97c6-jlwgh" (UID: "ece35fe8-abce-464f-a888-9798a5725a40") : secret "image-registry-tls" not found Sep 09 21:10:40.355691 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:40.355650 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:40.355884 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:40.355773 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:40.355884 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:40.355848 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle podName:05b70482-3a69-427f-8ce3-922c764c470b nodeName:}" failed. No retries permitted until 2026-09-09 21:10:56.355834482 +0000 UTC m=+66.462578289 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle") pod "router-default-64f7976768-h7vtq" (UID: "05b70482-3a69-427f-8ce3-922c764c470b") : configmap references non-existent config key: service-ca.crt Sep 09 21:10:40.356009 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:40.355937 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Sep 09 21:10:40.356009 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:40.356007 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs podName:05b70482-3a69-427f-8ce3-922c764c470b nodeName:}" failed. No retries permitted until 2026-09-09 21:10:56.355989234 +0000 UTC m=+66.462733053 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs") pod "router-default-64f7976768-h7vtq" (UID: "05b70482-3a69-427f-8ce3-922c764c470b") : secret "router-metrics-certs-default" not found Sep 09 21:10:40.456220 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:40.456187 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:40.456396 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:40.456335 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Sep 09 21:10:40.456396 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:40.456395 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls podName:ca9ee75e-2df9-47f5-b556-977f41f7eeb1 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:56.456379126 +0000 UTC m=+66.563122946 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls") pod "dns-default-qxszf" (UID: "ca9ee75e-2df9-47f5-b556-977f41f7eeb1") : secret "dns-default-metrics-tls" not found Sep 09 21:10:42.070518 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:42.070473 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:42.070951 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:42.070584 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:42.070951 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:42.070733 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Sep 09 21:10:42.070951 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:42.070801 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls podName:9e908db9-3103-49d9-a462-b3bc3757fd89 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:50.070785431 +0000 UTC m=+60.177529239 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls") pod "insights-runtime-extractor-4ckvp" (UID: "9e908db9-3103-49d9-a462-b3bc3757fd89") : secret "insights-runtime-extractor-tls" not found Sep 09 21:10:42.071074 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:42.071047 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e908db9-3103-49d9-a462-b3bc3757fd89-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:42.777723 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:42.777669 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-9hd77\" (UID: \"7bd30f8a-b08d-444d-b317-a32e7e0152b3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:42.777993 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:42.777823 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Sep 09 21:10:42.777993 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:42.777891 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert podName:7bd30f8a-b08d-444d-b317-a32e7e0152b3 nodeName:}" failed. No retries permitted until 2026-09-09 21:10:58.777876482 +0000 UTC m=+68.884620302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-9hd77" (UID: "7bd30f8a-b08d-444d-b317-a32e7e0152b3") : secret "networking-console-plugin-cert" not found Sep 09 21:10:44.709488 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:44.709450 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:44.709925 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:44.709508 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:44.709925 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:44.709826 2553 scope.go:117] "RemoveContainer" containerID="52f1acf97426c26f62fd41fc0a8a098bb6158b86c5bde453a8297ec2c3d8cf36" Sep 09 21:10:44.851636 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:44.851609 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:10:44.852041 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:44.852025 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/1.log" Sep 09 21:10:44.852124 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:44.852059 2553 generic.go:356] "Generic (PLEG): container finished" podID="6cec47f9-1808-4396-9ee3-4fc7c5cf1586" containerID="e42c5ce031c450b8da19890fa02dcd88f8a8df7d47850cf90a031c1befe40685" exitCode=255 Sep 09 21:10:44.852177 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:44.852136 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" event={"ID":"6cec47f9-1808-4396-9ee3-4fc7c5cf1586","Type":"ContainerDied","Data":"e42c5ce031c450b8da19890fa02dcd88f8a8df7d47850cf90a031c1befe40685"} Sep 09 21:10:44.852212 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:44.852191 2553 scope.go:117] "RemoveContainer" containerID="52f1acf97426c26f62fd41fc0a8a098bb6158b86c5bde453a8297ec2c3d8cf36" Sep 09 21:10:44.852580 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:44.852566 2553 scope.go:117] "RemoveContainer" containerID="e42c5ce031c450b8da19890fa02dcd88f8a8df7d47850cf90a031c1befe40685" Sep 09 21:10:44.852723 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:44.852705 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-9gxmg_openshift-console-operator(6cec47f9-1808-4396-9ee3-4fc7c5cf1586)\"" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" podUID="6cec47f9-1808-4396-9ee3-4fc7c5cf1586" Sep 09 21:10:45.857238 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:45.857204 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:10:50.140077 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:50.140039 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:50.142371 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:50.142345 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e908db9-3103-49d9-a462-b3bc3757fd89-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4ckvp\" (UID: \"9e908db9-3103-49d9-a462-b3bc3757fd89\") " pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:50.246621 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:50.246588 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4ckvp" Sep 09 21:10:50.528921 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:50.528885 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4ckvp"] Sep 09 21:10:50.536179 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:50.536149 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e908db9_3103_49d9_a462_b3bc3757fd89.slice/crio-feada501109846559087a189f5f264969a98f6d50d664d766bee87cea9b9cdc0 WatchSource:0}: Error finding container feada501109846559087a189f5f264969a98f6d50d664d766bee87cea9b9cdc0: Status 404 returned error can't find the container with id feada501109846559087a189f5f264969a98f6d50d664d766bee87cea9b9cdc0 Sep 09 21:10:50.802485 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:50.802407 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zmmsz" Sep 09 21:10:50.876324 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:50.876286 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4ckvp" event={"ID":"9e908db9-3103-49d9-a462-b3bc3757fd89","Type":"ContainerStarted","Data":"a5146e642a2be9e3c11a23c71a11afc0cdf212be9c3ed285837f2284037fb7f4"} Sep 09 21:10:50.876490 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:50.876345 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4ckvp" event={"ID":"9e908db9-3103-49d9-a462-b3bc3757fd89","Type":"ContainerStarted","Data":"feada501109846559087a189f5f264969a98f6d50d664d766bee87cea9b9cdc0"} Sep 09 21:10:51.881893 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:51.881855 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4ckvp" event={"ID":"9e908db9-3103-49d9-a462-b3bc3757fd89","Type":"ContainerStarted","Data":"354f935fc5656561f1093b3ae4f67255bcc37cbb30dca6d45bb710e551712192"} Sep 09 21:10:52.887486 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:52.887389 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4ckvp" event={"ID":"9e908db9-3103-49d9-a462-b3bc3757fd89","Type":"ContainerStarted","Data":"7b1bf4d46852ff8c37059da15cac184dde80c498ca107a6677ad139077186a82"} Sep 09 21:10:52.913428 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:52.913373 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4ckvp" podStartSLOduration=17.046249752 podStartE2EDuration="18.913358264s" totalImagesPullingTime="1.867108512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:10:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:52.912335315 +0000 UTC m=+63.019079162" watchObservedRunningTime="2026-09-09 21:10:52.913358264 +0000 UTC m=+63.020102087" Sep 09 21:10:54.709548 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:54.709502 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:54.709945 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:54.709570 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:10:54.709945 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:54.709862 2553 scope.go:117] "RemoveContainer" containerID="e42c5ce031c450b8da19890fa02dcd88f8a8df7d47850cf90a031c1befe40685" Sep 09 21:10:54.710033 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:10:54.710011 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-9gxmg_openshift-console-operator(6cec47f9-1808-4396-9ee3-4fc7c5cf1586)\"" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" podUID="6cec47f9-1808-4396-9ee3-4fc7c5cf1586" Sep 09 21:10:56.196038 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.195994 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:56.196038 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.196045 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl58d\" (UniqueName: \"kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d\") pod \"network-check-target-vk8h4\" (UID: \"7ed0ff95-e5da-49df-b563-a0988e9e8fe3\") " pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:56.196509 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.196068 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:56.198600 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.198580 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl58d\" (UniqueName: \"kubernetes.io/projected/7ed0ff95-e5da-49df-b563-a0988e9e8fe3-kube-api-access-wl58d\") pod \"network-check-target-vk8h4\" (UID: \"7ed0ff95-e5da-49df-b563-a0988e9e8fe3\") " pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:56.199143 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.199129 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Sep 09 21:10:56.200190 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.200173 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Sep 09 21:10:56.209203 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.209179 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a77a1afc-9eda-42b4-857c-2967b6d149d8-original-pull-secret\") pod \"global-pull-secret-syncer-tbgn7\" (UID: \"a77a1afc-9eda-42b4-857c-2967b6d149d8\") " pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:56.209317 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.209206 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1abf2101-20e2-4e99-bb20-206110f86504-metrics-certs\") pod \"network-metrics-daemon-55b8r\" (UID: \"1abf2101-20e2-4e99-bb20-206110f86504\") " pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:56.220283 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.220264 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ck66v\"" Sep 09 21:10:56.227228 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.227212 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-l7j9l\"" Sep 09 21:10:56.227228 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.227224 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-55b8r" Sep 09 21:10:56.234874 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.234857 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:56.296981 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.296938 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-f6tbw\" (UID: \"24725293-ce93-4439-92ff-0310c29b8e96\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:56.297156 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.297000 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:56.297156 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.297073 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:56.300255 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.300163 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/24725293-ce93-4439-92ff-0310c29b8e96-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-f6tbw\" (UID: \"24725293-ce93-4439-92ff-0310c29b8e96\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:56.300255 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.300173 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls\") pod \"image-registry-94b4b97c6-jlwgh\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:56.300255 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.300163 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/4cfb52c0-2216-418e-adf2-80859d555aa9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-6pg2n\" (UID: \"4cfb52c0-2216-418e-adf2-80859d555aa9\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:56.397996 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.397958 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:56.398193 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.398014 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:56.398558 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.398541 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/05b70482-3a69-427f-8ce3-922c764c470b-service-ca-bundle\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:56.400199 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.400181 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05b70482-3a69-427f-8ce3-922c764c470b-metrics-certs\") pod \"router-default-64f7976768-h7vtq\" (UID: \"05b70482-3a69-427f-8ce3-922c764c470b\") " pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:56.465313 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.465247 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xkzkp\"" Sep 09 21:10:56.473325 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.473308 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:56.475997 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.475982 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-bpjcw\"" Sep 09 21:10:56.483956 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.483936 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" Sep 09 21:10:56.499082 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.499054 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:56.499590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.499566 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-55b8r"] Sep 09 21:10:56.500814 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.500602 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tbgn7" Sep 09 21:10:56.501549 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.501531 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ca9ee75e-2df9-47f5-b556-977f41f7eeb1-metrics-tls\") pod \"dns-default-qxszf\" (UID: \"ca9ee75e-2df9-47f5-b556-977f41f7eeb1\") " pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:56.502873 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:56.502851 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1abf2101_20e2_4e99_bb20_206110f86504.slice/crio-50665cdd1ebde51f19d9c961c46a0219c2d3b94a6eb3bb8a9994e027ee6a7cab WatchSource:0}: Error finding container 50665cdd1ebde51f19d9c961c46a0219c2d3b94a6eb3bb8a9994e027ee6a7cab: Status 404 returned error can't find the container with id 50665cdd1ebde51f19d9c961c46a0219c2d3b94a6eb3bb8a9994e027ee6a7cab Sep 09 21:10:56.505568 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.505551 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vk8h4"] Sep 09 21:10:56.510933 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:56.510911 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ed0ff95_e5da_49df_b563_a0988e9e8fe3.slice/crio-16efc270665229302127448acaf4a2d90d5b987478650ef0ede22002454c489b WatchSource:0}: Error finding container 16efc270665229302127448acaf4a2d90d5b987478650ef0ede22002454c489b: Status 404 returned error can't find the container with id 16efc270665229302127448acaf4a2d90d5b987478650ef0ede22002454c489b Sep 09 21:10:56.543651 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.541671 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-gtnjz\"" Sep 09 21:10:56.543651 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.541938 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-n75x8\"" Sep 09 21:10:56.548453 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.548223 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" Sep 09 21:10:56.549785 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.549720 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:56.654182 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.654154 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7flhb\"" Sep 09 21:10:56.658321 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.658277 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qxszf" Sep 09 21:10:56.759335 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.759313 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-94b4b97c6-jlwgh"] Sep 09 21:10:56.761690 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:56.761666 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece35fe8_abce_464f_a888_9798a5725a40.slice/crio-c83a36e4c58b71af63fced79a43525067e9654e1697178ae143385fd9d73a87b WatchSource:0}: Error finding container c83a36e4c58b71af63fced79a43525067e9654e1697178ae143385fd9d73a87b: Status 404 returned error can't find the container with id c83a36e4c58b71af63fced79a43525067e9654e1697178ae143385fd9d73a87b Sep 09 21:10:56.762302 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.762285 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw"] Sep 09 21:10:56.797565 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.797540 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tbgn7"] Sep 09 21:10:56.801045 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:56.801007 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda77a1afc_9eda_42b4_857c_2967b6d149d8.slice/crio-633ef6383384c708e50488716fbcbd6465608825b8498d6838208b3dabe3ed16 WatchSource:0}: Error finding container 633ef6383384c708e50488716fbcbd6465608825b8498d6838208b3dabe3ed16: Status 404 returned error can't find the container with id 633ef6383384c708e50488716fbcbd6465608825b8498d6838208b3dabe3ed16 Sep 09 21:10:56.846622 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.846598 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n"] Sep 09 21:10:56.848861 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:56.848824 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cfb52c0_2216_418e_adf2_80859d555aa9.slice/crio-71a08a7dbe2ebb6c3c36d8b068113ae4b2e0f65bbfa94e3fa39ef8a8576def8f WatchSource:0}: Error finding container 71a08a7dbe2ebb6c3c36d8b068113ae4b2e0f65bbfa94e3fa39ef8a8576def8f: Status 404 returned error can't find the container with id 71a08a7dbe2ebb6c3c36d8b068113ae4b2e0f65bbfa94e3fa39ef8a8576def8f Sep 09 21:10:56.850143 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.850118 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-64f7976768-h7vtq"] Sep 09 21:10:56.851910 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:56.851873 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05b70482_3a69_427f_8ce3_922c764c470b.slice/crio-3ea2e11c31362fdae7c39fe1299c478f651b21bb04fea360ce0cc6d46dede507 WatchSource:0}: Error finding container 3ea2e11c31362fdae7c39fe1299c478f651b21bb04fea360ce0cc6d46dede507: Status 404 returned error can't find the container with id 3ea2e11c31362fdae7c39fe1299c478f651b21bb04fea360ce0cc6d46dede507 Sep 09 21:10:56.902772 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.902719 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" event={"ID":"4cfb52c0-2216-418e-adf2-80859d555aa9","Type":"ContainerStarted","Data":"71a08a7dbe2ebb6c3c36d8b068113ae4b2e0f65bbfa94e3fa39ef8a8576def8f"} Sep 09 21:10:56.904303 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.904254 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tbgn7" event={"ID":"a77a1afc-9eda-42b4-857c-2967b6d149d8","Type":"ContainerStarted","Data":"633ef6383384c708e50488716fbcbd6465608825b8498d6838208b3dabe3ed16"} Sep 09 21:10:56.905683 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.905652 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64f7976768-h7vtq" event={"ID":"05b70482-3a69-427f-8ce3-922c764c470b","Type":"ContainerStarted","Data":"3ea2e11c31362fdae7c39fe1299c478f651b21bb04fea360ce0cc6d46dede507"} Sep 09 21:10:56.907393 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.907368 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" event={"ID":"24725293-ce93-4439-92ff-0310c29b8e96","Type":"ContainerStarted","Data":"80ffbba0e68456b96b93a038c9a620a93cbb2901655bd720a000c1f9548e12fe"} Sep 09 21:10:56.909207 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.909168 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerStarted","Data":"6bbf4e4823cad046b1137375660d9f9c245ff167c091bd397e38c11f69fd57a5"} Sep 09 21:10:56.909372 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.909353 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerStarted","Data":"c83a36e4c58b71af63fced79a43525067e9654e1697178ae143385fd9d73a87b"} Sep 09 21:10:56.910067 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.910035 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:10:56.911545 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.911523 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-55b8r" event={"ID":"1abf2101-20e2-4e99-bb20-206110f86504","Type":"ContainerStarted","Data":"50665cdd1ebde51f19d9c961c46a0219c2d3b94a6eb3bb8a9994e027ee6a7cab"} Sep 09 21:10:56.913487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.913460 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vk8h4" event={"ID":"7ed0ff95-e5da-49df-b563-a0988e9e8fe3","Type":"ContainerStarted","Data":"bfad218a4f77ece05a820aef9ce708daf3043d88fcbe6fafba2709c5c6f020da"} Sep 09 21:10:56.913604 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.913498 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vk8h4" event={"ID":"7ed0ff95-e5da-49df-b563-a0988e9e8fe3","Type":"ContainerStarted","Data":"16efc270665229302127448acaf4a2d90d5b987478650ef0ede22002454c489b"} Sep 09 21:10:56.913604 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.913533 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:10:56.934397 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.934318 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qxszf"] Sep 09 21:10:56.935964 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.935916 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podStartSLOduration=65.935897267 podStartE2EDuration="1m5.935897267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:56.934923177 +0000 UTC m=+67.041667026" watchObservedRunningTime="2026-09-09 21:10:56.935897267 +0000 UTC m=+67.042641094" Sep 09 21:10:56.938976 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:10:56.938947 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca9ee75e_2df9_47f5_b556_977f41f7eeb1.slice/crio-c10af1219f9cda53b0ab46c334f6b2ad6b793c58e46d9a43f481a088ffd0cd27 WatchSource:0}: Error finding container c10af1219f9cda53b0ab46c334f6b2ad6b793c58e46d9a43f481a088ffd0cd27: Status 404 returned error can't find the container with id c10af1219f9cda53b0ab46c334f6b2ad6b793c58e46d9a43f481a088ffd0cd27 Sep 09 21:10:56.953055 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:56.952903 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vk8h4" podStartSLOduration=66.952880753 podStartE2EDuration="1m6.952880753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:56.951992807 +0000 UTC m=+67.058736636" watchObservedRunningTime="2026-09-09 21:10:56.952880753 +0000 UTC m=+67.059624590" Sep 09 21:10:57.921348 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:57.921312 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-64f7976768-h7vtq" event={"ID":"05b70482-3a69-427f-8ce3-922c764c470b","Type":"ContainerStarted","Data":"a9616c7c2b5226d07552cd15108b1517187fbaadbfe86c4b9ef2133571198611"} Sep 09 21:10:57.925273 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:57.925223 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-55b8r" event={"ID":"1abf2101-20e2-4e99-bb20-206110f86504","Type":"ContainerStarted","Data":"02e5789cfeec073d6b1a830aeea47d955d154e41323b86c231c7ae0099b6ae0c"} Sep 09 21:10:57.925437 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:57.925283 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-55b8r" event={"ID":"1abf2101-20e2-4e99-bb20-206110f86504","Type":"ContainerStarted","Data":"3bd4d3de3ceb23b75108daa673f920c8e08458cb5d352107f580f6c888500882"} Sep 09 21:10:57.927814 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:57.927780 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qxszf" event={"ID":"ca9ee75e-2df9-47f5-b556-977f41f7eeb1","Type":"ContainerStarted","Data":"c10af1219f9cda53b0ab46c334f6b2ad6b793c58e46d9a43f481a088ffd0cd27"} Sep 09 21:10:57.942731 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:57.942226 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-64f7976768-h7vtq" podStartSLOduration=37.942205855 podStartE2EDuration="37.942205855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:57.940613965 +0000 UTC m=+68.047357807" watchObservedRunningTime="2026-09-09 21:10:57.942205855 +0000 UTC m=+68.048949681" Sep 09 21:10:57.957982 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:57.957038 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-55b8r" podStartSLOduration=66.078649364 podStartE2EDuration="1m6.957017448s" totalImagesPullingTime="878.368084ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:10:57.955967842 +0000 UTC m=+68.062711669" watchObservedRunningTime="2026-09-09 21:10:57.957017448 +0000 UTC m=+68.063761275" Sep 09 21:10:58.550911 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:58.550877 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:58.554212 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:58.554179 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:58.824991 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:58.824905 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-9hd77\" (UID: \"7bd30f8a-b08d-444d-b317-a32e7e0152b3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:58.827972 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:58.827918 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7bd30f8a-b08d-444d-b317-a32e7e0152b3-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-9hd77\" (UID: \"7bd30f8a-b08d-444d-b317-a32e7e0152b3\") " pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:10:58.932626 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:58.932562 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:58.933859 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:58.933830 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-64f7976768-h7vtq" Sep 09 21:10:58.972357 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:58.972323 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zm5n7\"" Sep 09 21:10:58.980294 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:10:58.980259 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" Sep 09 21:11:01.765480 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:01.765386 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-9hd77"] Sep 09 21:11:01.816400 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:11:01.816365 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd30f8a_b08d_444d_b317_a32e7e0152b3.slice/crio-1cf847fe4c51367c880dc4463b94821e8ec1f6992bb7de9316b815c6f647ac05 WatchSource:0}: Error finding container 1cf847fe4c51367c880dc4463b94821e8ec1f6992bb7de9316b815c6f647ac05: Status 404 returned error can't find the container with id 1cf847fe4c51367c880dc4463b94821e8ec1f6992bb7de9316b815c6f647ac05 Sep 09 21:11:01.951677 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:01.951603 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" event={"ID":"24725293-ce93-4439-92ff-0310c29b8e96","Type":"ContainerStarted","Data":"f716f8d66511a4627702b509b46a0e2d21214af4f7bf6dce2e939bbd6e1c21a2"} Sep 09 21:11:01.964832 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:01.961317 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qxszf" event={"ID":"ca9ee75e-2df9-47f5-b556-977f41f7eeb1","Type":"ContainerStarted","Data":"df0bf53ca2625128c6a2f4ae12515d243dda43901c5fa7fff1fbc4df964ad12b"} Sep 09 21:11:01.966536 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:01.966505 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" event={"ID":"7bd30f8a-b08d-444d-b317-a32e7e0152b3","Type":"ContainerStarted","Data":"1cf847fe4c51367c880dc4463b94821e8ec1f6992bb7de9316b815c6f647ac05"} Sep 09 21:11:01.968575 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:01.968553 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" event={"ID":"4cfb52c0-2216-418e-adf2-80859d555aa9","Type":"ContainerStarted","Data":"530ac9fdbd4801b890d0c1fb4d8cd85c7ae8d6dfab9b73534780800a92e4eb4d"} Sep 09 21:11:01.986708 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:01.986328 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-6pg2n" podStartSLOduration=37.335878421 podStartE2EDuration="41.986305285s" totalImagesPullingTime="4.650426864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:01.98625851 +0000 UTC m=+72.093002355" watchObservedRunningTime="2026-09-09 21:11:01.986305285 +0000 UTC m=+72.093049117" Sep 09 21:11:02.973230 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:02.973195 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tbgn7" event={"ID":"a77a1afc-9eda-42b4-857c-2967b6d149d8","Type":"ContainerStarted","Data":"92cf296d676dd7d258287e34d415d633789cc6d8da571e1964409463ae9232f8"} Sep 09 21:11:02.975069 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:02.975049 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" event={"ID":"24725293-ce93-4439-92ff-0310c29b8e96","Type":"ContainerStarted","Data":"f2aadd88d6f725ee7a8e89544362727c99147dd7086916f3ff4872e48fc63178"} Sep 09 21:11:02.976881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:02.976861 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qxszf" event={"ID":"ca9ee75e-2df9-47f5-b556-977f41f7eeb1","Type":"ContainerStarted","Data":"2d3ad626a1d584da8eac1da7defd67c94b37f8671e38109a5379d84ff789f4be"} Sep 09 21:11:02.977149 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:02.977132 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qxszf" Sep 09 21:11:02.991342 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:02.991297 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tbgn7" podStartSLOduration=66.941752394 podStartE2EDuration="1m11.991281596s" totalImagesPullingTime="5.049529202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:02.990249326 +0000 UTC m=+73.096993178" watchObservedRunningTime="2026-09-09 21:11:02.991281596 +0000 UTC m=+73.098025421" Sep 09 21:11:03.008428 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:03.008381 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qxszf" podStartSLOduration=34.451514499 podStartE2EDuration="39.008364656s" totalImagesPullingTime="4.556850157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:03.007495535 +0000 UTC m=+73.114239380" watchObservedRunningTime="2026-09-09 21:11:03.008364656 +0000 UTC m=+73.115108481" Sep 09 21:11:03.026435 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:03.026378 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw" podStartSLOduration=38.325772509 podStartE2EDuration="43.026361396s" totalImagesPullingTime="4.700588887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:03.025871623 +0000 UTC m=+73.132615468" watchObservedRunningTime="2026-09-09 21:11:03.026361396 +0000 UTC m=+73.133105222" Sep 09 21:11:03.981452 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:03.981417 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" event={"ID":"7bd30f8a-b08d-444d-b317-a32e7e0152b3","Type":"ContainerStarted","Data":"b57be71d9ba24895dcef1a79f612a05f76942cbf5aa0fb9012fbf94f4855c959"} Sep 09 21:11:04.002529 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:04.002479 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-9hd77" podStartSLOduration=36.497042677 podStartE2EDuration="38.00246172s" totalImagesPullingTime="1.505419043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:04.001871445 +0000 UTC m=+74.108615275" watchObservedRunningTime="2026-09-09 21:11:04.00246172 +0000 UTC m=+74.109205546" Sep 09 21:11:05.383539 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:05.383504 2553 scope.go:117] "RemoveContainer" containerID="e42c5ce031c450b8da19890fa02dcd88f8a8df7d47850cf90a031c1befe40685" Sep 09 21:11:05.990848 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:05.990817 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:11:05.991033 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:05.990894 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" event={"ID":"6cec47f9-1808-4396-9ee3-4fc7c5cf1586","Type":"ContainerStarted","Data":"2baa414379851f61ff53cbced0e766c5150574240078251b2a7afdc4c7e0ec97"} Sep 09 21:11:05.991181 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:05.991162 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:11:06.008404 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:06.008349 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" podStartSLOduration=38.31631783 podStartE2EDuration="46.008333763s" totalImagesPullingTime="7.692015933s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:11:06.008055178 +0000 UTC m=+76.114799022" watchObservedRunningTime="2026-09-09 21:11:06.008333763 +0000 UTC m=+76.115077589" Sep 09 21:11:06.074221 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:06.074190 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-9gxmg" Sep 09 21:11:12.983780 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:12.983676 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qxszf" Sep 09 21:11:16.478248 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:16.478208 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:16.478725 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:16.478272 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:18.936323 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:18.936289 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:18.936695 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:18.936345 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:26.477506 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:26.477460 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:26.477970 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:26.477531 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:27.930539 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:27.930504 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vk8h4" Sep 09 21:11:28.936580 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:28.936546 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:28.936941 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:28.936599 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:36.477987 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:36.477950 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:36.478404 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:36.478012 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:36.478404 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:36.478054 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:11:36.478632 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:36.478598 2553 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"6bbf4e4823cad046b1137375660d9f9c245ff167c091bd397e38c11f69fd57a5"} pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" containerMessage="Container registry failed liveness probe, will be restarted" Sep 09 21:11:36.482458 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:36.482432 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:36.482585 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:36.482480 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:46.482818 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:46.482784 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:46.483238 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:46.482846 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:54.161404 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:54.161369 2553 generic.go:356] "Generic (PLEG): container finished" podID="bb25b5c9-2b1a-4286-9668-a57b10e163d6" containerID="8d9396e09ab6f7c278e36482171d176f7799e45fa6d541bff162bc012a547047" exitCode=0 Sep 09 21:11:54.161842 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:54.161450 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" event={"ID":"bb25b5c9-2b1a-4286-9668-a57b10e163d6","Type":"ContainerDied","Data":"8d9396e09ab6f7c278e36482171d176f7799e45fa6d541bff162bc012a547047"} Sep 09 21:11:54.161842 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:54.161770 2553 scope.go:117] "RemoveContainer" containerID="8d9396e09ab6f7c278e36482171d176f7799e45fa6d541bff162bc012a547047" Sep 09 21:11:55.166545 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:55.166506 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-7vkst" event={"ID":"bb25b5c9-2b1a-4286-9668-a57b10e163d6","Type":"ContainerStarted","Data":"743b8f6cc3c2d80ea86f3825490477da0d85139bbcfe33114cd124499625cfcf"} Sep 09 21:11:56.482870 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:56.482837 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:11:56.483289 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:56.482889 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:11:59.181177 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:59.181139 2553 generic.go:356] "Generic (PLEG): container finished" podID="0a512f34-bdcf-44ee-a4bd-2c2b49cb7553" containerID="16900cdcf88bec1d829b1084c3e31b73b387619c0bcd989f192142bc05f8506d" exitCode=0 Sep 09 21:11:59.181533 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:59.181190 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" event={"ID":"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553","Type":"ContainerDied","Data":"16900cdcf88bec1d829b1084c3e31b73b387619c0bcd989f192142bc05f8506d"} Sep 09 21:11:59.181533 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:11:59.181464 2553 scope.go:117] "RemoveContainer" containerID="16900cdcf88bec1d829b1084c3e31b73b387619c0bcd989f192142bc05f8506d" Sep 09 21:12:00.186167 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:00.186126 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-hghqt" event={"ID":"0a512f34-bdcf-44ee-a4bd-2c2b49cb7553","Type":"ContainerStarted","Data":"7ce69fa3c173e1729cb5cbbe5ce5803176bd6799d413994c9a785169fd78bac2"} Sep 09 21:12:01.498082 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:01.498044 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" containerID="cri-o://6bbf4e4823cad046b1137375660d9f9c245ff167c091bd397e38c11f69fd57a5" gracePeriod=30 Sep 09 21:12:02.194288 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:02.194252 2553 generic.go:356] "Generic (PLEG): container finished" podID="ece35fe8-abce-464f-a888-9798a5725a40" containerID="6bbf4e4823cad046b1137375660d9f9c245ff167c091bd397e38c11f69fd57a5" exitCode=0 Sep 09 21:12:02.194453 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:02.194310 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerDied","Data":"6bbf4e4823cad046b1137375660d9f9c245ff167c091bd397e38c11f69fd57a5"} Sep 09 21:12:02.194453 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:02.194347 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerStarted","Data":"f2cbe3131c83dd1a299bdd340ab12dc1b7d835d75f583197424e484b79bdce94"} Sep 09 21:12:02.194453 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:02.194411 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:12:09.220532 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:09.220496 2553 generic.go:356] "Generic (PLEG): container finished" podID="0896f595-88a0-4e6e-956a-6abd1cbc0f97" containerID="f738b5163a0150e21a22d1b1695b59eafcdab7cfc43e5a2e7c0f6c0989be5ec0" exitCode=0 Sep 09 21:12:09.220911 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:09.220545 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" event={"ID":"0896f595-88a0-4e6e-956a-6abd1cbc0f97","Type":"ContainerDied","Data":"f738b5163a0150e21a22d1b1695b59eafcdab7cfc43e5a2e7c0f6c0989be5ec0"} Sep 09 21:12:09.220911 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:09.220831 2553 scope.go:117] "RemoveContainer" containerID="f738b5163a0150e21a22d1b1695b59eafcdab7cfc43e5a2e7c0f6c0989be5ec0" Sep 09 21:12:10.225724 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:10.225690 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-gstjf" event={"ID":"0896f595-88a0-4e6e-956a-6abd1cbc0f97","Type":"ContainerStarted","Data":"ac8dc093254fcb84a67096f8666117fa083f582f328c3230b15b91a0d292eee9"} Sep 09 21:12:16.477599 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:16.477562 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:16.478000 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:16.477612 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:23.202377 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:23.202337 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:23.202753 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:23.202388 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:26.477782 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:26.477746 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:26.478257 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:26.477799 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:33.202714 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:33.202676 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:33.203192 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:33.202736 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:36.477967 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:36.477925 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:36.478358 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:36.478011 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:36.478358 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:36.478048 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:12:36.478519 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:36.478503 2553 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"f2cbe3131c83dd1a299bdd340ab12dc1b7d835d75f583197424e484b79bdce94"} pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" containerMessage="Container registry failed liveness probe, will be restarted" Sep 09 21:12:36.482047 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:36.482019 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:36.482187 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:36.482068 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:46.482207 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:46.482126 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:46.482207 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:46.482197 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:12:56.482271 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:56.482233 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:12:56.482744 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:12:56.482291 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:01.497534 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:01.497487 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" containerID="cri-o://f2cbe3131c83dd1a299bdd340ab12dc1b7d835d75f583197424e484b79bdce94" gracePeriod=30 Sep 09 21:13:02.415758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:02.415722 2553 generic.go:356] "Generic (PLEG): container finished" podID="ece35fe8-abce-464f-a888-9798a5725a40" containerID="f2cbe3131c83dd1a299bdd340ab12dc1b7d835d75f583197424e484b79bdce94" exitCode=0 Sep 09 21:13:02.416052 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:02.415776 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerDied","Data":"f2cbe3131c83dd1a299bdd340ab12dc1b7d835d75f583197424e484b79bdce94"} Sep 09 21:13:02.416052 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:02.415824 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerStarted","Data":"df64e398e462ec0a4196fd398a011c8a7af7d42b99f9b6caf5caca5941473154"} Sep 09 21:13:02.416052 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:02.415850 2553 scope.go:117] "RemoveContainer" containerID="6bbf4e4823cad046b1137375660d9f9c245ff167c091bd397e38c11f69fd57a5" Sep 09 21:13:02.416052 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:02.415981 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:13:16.477635 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:16.477597 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:16.478000 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:16.477644 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:23.424940 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:23.424899 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:23.425334 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:23.424961 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:26.477309 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:26.477273 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:26.477701 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:26.477329 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:33.425253 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:33.425217 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:33.425704 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:33.425272 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:36.478061 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:36.478024 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:36.478555 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:36.478095 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:36.478555 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:36.478158 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:13:36.478730 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:36.478702 2553 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"df64e398e462ec0a4196fd398a011c8a7af7d42b99f9b6caf5caca5941473154"} pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" containerMessage="Container registry failed liveness probe, will be restarted" Sep 09 21:13:36.482174 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:36.482149 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:36.482284 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:36.482192 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:46.482626 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:46.482590 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:46.483002 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:46.482648 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:13:56.483200 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:56.483165 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:13:56.483582 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:13:56.483225 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:01.497176 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:01.497135 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" containerID="cri-o://df64e398e462ec0a4196fd398a011c8a7af7d42b99f9b6caf5caca5941473154" gracePeriod=30 Sep 09 21:14:01.617722 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:01.617698 2553 generic.go:356] "Generic (PLEG): container finished" podID="ece35fe8-abce-464f-a888-9798a5725a40" containerID="df64e398e462ec0a4196fd398a011c8a7af7d42b99f9b6caf5caca5941473154" exitCode=0 Sep 09 21:14:01.617824 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:01.617739 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerDied","Data":"df64e398e462ec0a4196fd398a011c8a7af7d42b99f9b6caf5caca5941473154"} Sep 09 21:14:01.617824 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:01.617787 2553 scope.go:117] "RemoveContainer" containerID="f2cbe3131c83dd1a299bdd340ab12dc1b7d835d75f583197424e484b79bdce94" Sep 09 21:14:02.622997 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:02.622961 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerStarted","Data":"641c9ed9588b4ed29e6cd6a62d28bb4cf823de47a5c3bf39e98b38e16b997c99"} Sep 09 21:14:02.623502 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:02.623087 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:14:16.477860 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:16.477769 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:16.477860 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:16.477831 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:23.630902 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:23.630865 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:23.631293 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:23.630922 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:26.477139 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:26.477084 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:26.477514 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:26.477154 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:33.630336 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:33.630301 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:33.630747 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:33.630360 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:36.477245 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:36.477206 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:36.477726 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:36.477266 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:36.477726 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:36.477312 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:14:36.477899 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:36.477879 2553 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"641c9ed9588b4ed29e6cd6a62d28bb4cf823de47a5c3bf39e98b38e16b997c99"} pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" containerMessage="Container registry failed liveness probe, will be restarted" Sep 09 21:14:36.481328 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:36.481303 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:36.481461 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:36.481354 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:46.481930 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:46.481894 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:46.482318 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:46.481950 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:14:50.317651 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:50.317621 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:14:50.321927 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:50.321905 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:14:50.331961 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:50.331936 2553 kubelet.go:1648] "Image garbage collection succeeded" Sep 09 21:14:56.482430 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:56.482396 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:14:56.484667 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:14:56.482458 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:01.496335 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:01.496292 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" containerID="cri-o://641c9ed9588b4ed29e6cd6a62d28bb4cf823de47a5c3bf39e98b38e16b997c99" gracePeriod=30 Sep 09 21:15:01.824741 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:01.824655 2553 generic.go:356] "Generic (PLEG): container finished" podID="ece35fe8-abce-464f-a888-9798a5725a40" containerID="641c9ed9588b4ed29e6cd6a62d28bb4cf823de47a5c3bf39e98b38e16b997c99" exitCode=0 Sep 09 21:15:01.824741 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:01.824728 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerDied","Data":"641c9ed9588b4ed29e6cd6a62d28bb4cf823de47a5c3bf39e98b38e16b997c99"} Sep 09 21:15:01.824979 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:01.824781 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerStarted","Data":"e1f087d3c4c914880760cc8a974fa158329d097a282a1d117b8da744c10dd38f"} Sep 09 21:15:01.824979 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:01.824801 2553 scope.go:117] "RemoveContainer" containerID="df64e398e462ec0a4196fd398a011c8a7af7d42b99f9b6caf5caca5941473154" Sep 09 21:15:01.824979 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:01.824907 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:15:16.477766 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:16.477731 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:16.478172 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:16.477794 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:22.834541 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:22.834509 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:22.834974 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:22.834557 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:26.478217 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:26.478182 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:26.478605 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:26.478243 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:32.834223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:32.834189 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:32.834591 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:32.834250 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:36.477355 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:36.477322 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:36.477757 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:36.477372 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:36.477757 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:36.477405 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:15:36.477845 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:36.477805 2553 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"e1f087d3c4c914880760cc8a974fa158329d097a282a1d117b8da744c10dd38f"} pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" containerMessage="Container registry failed liveness probe, will be restarted" Sep 09 21:15:36.481243 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:36.481218 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:36.481371 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:36.481256 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:46.481881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:46.481800 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:46.481881 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:46.481853 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:15:56.482193 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:56.482157 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:15:56.482645 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:15:56.482212 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:01.496118 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:01.496056 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" containerID="cri-o://e1f087d3c4c914880760cc8a974fa158329d097a282a1d117b8da744c10dd38f" gracePeriod=30 Sep 09 21:16:02.038913 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:02.038879 2553 generic.go:356] "Generic (PLEG): container finished" podID="ece35fe8-abce-464f-a888-9798a5725a40" containerID="e1f087d3c4c914880760cc8a974fa158329d097a282a1d117b8da744c10dd38f" exitCode=0 Sep 09 21:16:02.039138 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:02.038925 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerDied","Data":"e1f087d3c4c914880760cc8a974fa158329d097a282a1d117b8da744c10dd38f"} Sep 09 21:16:02.039138 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:02.038993 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerStarted","Data":"959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525"} Sep 09 21:16:02.039138 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:02.039025 2553 scope.go:117] "RemoveContainer" containerID="641c9ed9588b4ed29e6cd6a62d28bb4cf823de47a5c3bf39e98b38e16b997c99" Sep 09 21:16:02.039316 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:02.039219 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:16:16.477133 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:16.477083 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:16.477583 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:16.477157 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:21.499982 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:21.499938 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 09 21:16:21.499982 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:21.499973 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found Sep 09 21:16:21.499982 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:21.499986 2553 projected.go:196] Error preparing data for projected volume kube-api-access-2lj75 for pod openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:21.500495 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:21.500046 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75 podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:16:22.000031677 +0000 UTC m=+392.106775481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2lj75" (UniqueName: "kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:22.003716 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:22.003680 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 09 21:16:22.003716 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:22.003715 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found Sep 09 21:16:22.003972 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:22.003728 2553 projected.go:196] Error preparing data for projected volume kube-api-access-2lj75 for pod openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:22.003972 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:22.003788 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75 podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:16:23.003771431 +0000 UTC m=+393.110515235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2lj75" (UniqueName: "kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:23.011341 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:23.011307 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 09 21:16:23.011341 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:23.011340 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found Sep 09 21:16:23.011341 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:23.011350 2553 projected.go:196] Error preparing data for projected volume kube-api-access-2lj75 for pod openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:23.011801 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:23.011402 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75 podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:16:25.011388881 +0000 UTC m=+395.118132686 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2lj75" (UniqueName: "kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:23.048067 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:23.048034 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:23.048214 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:23.048082 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:25.025760 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:25.025720 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 09 21:16:25.025760 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:25.025754 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found Sep 09 21:16:25.025760 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:25.025766 2553 projected.go:196] Error preparing data for projected volume kube-api-access-2lj75 for pod openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:25.026236 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:25.025816 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75 podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:16:29.025801137 +0000 UTC m=+399.132544941 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2lj75" (UniqueName: "kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:26.478327 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:26.478292 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:26.478710 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:26.478346 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:29.059391 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:29.059358 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 09 21:16:29.059391 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:29.059391 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found Sep 09 21:16:29.059391 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:29.059404 2553 projected.go:196] Error preparing data for projected volume kube-api-access-2lj75 for pod openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:29.060032 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:29.059479 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75 podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:16:37.059459746 +0000 UTC m=+407.166203552 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2lj75" (UniqueName: "kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:33.048200 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:33.048160 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:33.048575 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:33.048223 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:36.478163 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:36.478124 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:36.478611 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:36.478174 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:36.478611 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:36.478209 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:16:36.478712 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:36.478611 2553 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525"} pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" containerMessage="Container registry failed liveness probe, will be restarted" Sep 09 21:16:36.482155 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:36.482126 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:36.482321 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:36.482179 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:37.126825 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:37.126778 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 09 21:16:37.126825 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:37.126819 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found Sep 09 21:16:37.126825 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:37.126832 2553 projected.go:196] Error preparing data for projected volume kube-api-access-2lj75 for pod openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:37.127163 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:37.126900 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75 podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:16:53.126883676 +0000 UTC m=+423.233627480 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2lj75" (UniqueName: "kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:46.483211 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:46.483175 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:46.483590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:46.483237 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:16:53.151257 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:53.151216 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 09 21:16:53.151257 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:53.151251 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found Sep 09 21:16:53.151257 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:53.151263 2553 projected.go:196] Error preparing data for projected volume kube-api-access-2lj75 for pod openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:53.151793 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:16:53.151313 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75 podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:17:25.151299077 +0000 UTC m=+455.258042880 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2lj75" (UniqueName: "kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:16:56.482605 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:56.482570 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:16:56.482978 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:16:56.482623 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:17:01.497401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:17:01.497351 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" containerID="cri-o://959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525" gracePeriod=30 Sep 09 21:17:01.608500 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:17:01.608469 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-94b4b97c6-jlwgh_openshift-image-registry(ece35fe8-abce-464f-a888-9798a5725a40)\"" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" Sep 09 21:17:02.252477 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:17:02.252443 2553 generic.go:356] "Generic (PLEG): container finished" podID="ece35fe8-abce-464f-a888-9798a5725a40" containerID="959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525" exitCode=0 Sep 09 21:17:02.252737 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:17:02.252517 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerDied","Data":"959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525"} Sep 09 21:17:02.252737 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:17:02.252577 2553 scope.go:117] "RemoveContainer" containerID="e1f087d3c4c914880760cc8a974fa158329d097a282a1d117b8da744c10dd38f" Sep 09 21:17:02.252951 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:17:02.252930 2553 scope.go:117] "RemoveContainer" containerID="959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525" Sep 09 21:17:02.253146 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:17:02.253127 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-94b4b97c6-jlwgh_openshift-image-registry(ece35fe8-abce-464f-a888-9798a5725a40)\"" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" Sep 09 21:17:16.383726 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:17:16.383645 2553 scope.go:117] "RemoveContainer" containerID="959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525" Sep 09 21:17:16.384129 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:17:16.383813 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-94b4b97c6-jlwgh_openshift-image-registry(ece35fe8-abce-464f-a888-9798a5725a40)\"" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" Sep 09 21:17:25.204572 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:17:25.204533 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 09 21:17:25.204572 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:17:25.204568 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found Sep 09 21:17:25.204572 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:17:25.204577 2553 projected.go:196] Error preparing data for projected volume kube-api-access-2lj75 for pod openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:17:25.205054 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:17:25.204638 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75 podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:18:29.204624735 +0000 UTC m=+519.311368539 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2lj75" (UniqueName: "kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:17:28.383766 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:17:28.383732 2553 scope.go:117] "RemoveContainer" containerID="959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525" Sep 09 21:17:28.384271 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:17:28.383927 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-94b4b97c6-jlwgh_openshift-image-registry(ece35fe8-abce-464f-a888-9798a5725a40)\"" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" Sep 09 21:17:40.386331 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:17:40.386301 2553 scope.go:117] "RemoveContainer" containerID="959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525" Sep 09 21:17:40.386954 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:17:40.386498 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-94b4b97c6-jlwgh_openshift-image-registry(ece35fe8-abce-464f-a888-9798a5725a40)\"" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" Sep 09 21:17:53.384209 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:17:53.384177 2553 scope.go:117] "RemoveContainer" containerID="959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525" Sep 09 21:17:53.384580 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:17:53.384329 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-94b4b97c6-jlwgh_openshift-image-registry(ece35fe8-abce-464f-a888-9798a5725a40)\"" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" Sep 09 21:18:05.383671 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:05.383641 2553 scope.go:117] "RemoveContainer" containerID="959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525" Sep 09 21:18:05.384078 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:18:05.383791 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-94b4b97c6-jlwgh_openshift-image-registry(ece35fe8-abce-464f-a888-9798a5725a40)\"" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" Sep 09 21:18:19.384148 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:19.384086 2553 scope.go:117] "RemoveContainer" containerID="959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525" Sep 09 21:18:19.384622 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:18:19.384283 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"registry\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=registry pod=image-registry-94b4b97c6-jlwgh_openshift-image-registry(ece35fe8-abce-464f-a888-9798a5725a40)\"" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" Sep 09 21:18:29.304548 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:18:29.304495 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/kube-root-ca.crt: configmap "kube-root-ca.crt" not found Sep 09 21:18:29.304548 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:18:29.304545 2553 projected.go:291] Couldn't get configMap openshift-cluster-samples-operator/openshift-service-ca.crt: configmap "openshift-service-ca.crt" not found Sep 09 21:18:29.304548 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:18:29.304556 2553 projected.go:196] Error preparing data for projected volume kube-api-access-2lj75 for pod openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-f6tbw: [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:18:29.305019 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:18:29.304635 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75 podName:24725293-ce93-4439-92ff-0310c29b8e96 nodeName:}" failed. No retries permitted until 2026-09-09 21:20:31.30462068 +0000 UTC m=+641.411364485 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2lj75" (UniqueName: "kubernetes.io/projected/24725293-ce93-4439-92ff-0310c29b8e96-kube-api-access-2lj75") pod "cluster-samples-operator-7cdcfcf76c-f6tbw" (UID: "24725293-ce93-4439-92ff-0310c29b8e96") : [configmap "kube-root-ca.crt" not found, configmap "openshift-service-ca.crt" not found] Sep 09 21:18:32.383836 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:32.383800 2553 scope.go:117] "RemoveContainer" containerID="959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525" Sep 09 21:18:32.563698 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:32.563656 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerStarted","Data":"0e85901e0972abc0f701aaf6ed77c73ece10ed44c7ca9bce0e71fee2e5493422"} Sep 09 21:18:32.563890 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:32.563862 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:18:46.479339 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:46.479255 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:18:46.479339 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:46.479323 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:18:53.572049 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.572012 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz"] Sep 09 21:18:53.574213 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.574185 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:18:53.574491 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.574458 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:18:53.576302 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.576279 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:18:53.579304 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.579260 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Sep 09 21:18:53.580243 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.580225 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Sep 09 21:18:53.580427 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.580407 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Sep 09 21:18:53.580502 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.580487 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Sep 09 21:18:53.588402 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.588374 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz"] Sep 09 21:18:53.596323 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.596294 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7ff4d48dfc-cnc6f"] Sep 09 21:18:53.599338 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.599322 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.603359 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.603336 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-cp7lf"] Sep 09 21:18:53.606353 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.606332 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-cp7lf" Sep 09 21:18:53.607014 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.606992 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Sep 09 21:18:53.607322 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.607302 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Sep 09 21:18:53.608753 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.608737 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-67778\"" Sep 09 21:18:53.608906 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.608885 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Sep 09 21:18:53.609896 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.609868 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9pr5v\"" Sep 09 21:18:53.610035 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.609925 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Sep 09 21:18:53.610160 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.610143 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Sep 09 21:18:53.610763 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.610742 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Sep 09 21:18:53.614276 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.614253 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Sep 09 21:18:53.615940 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.615920 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Sep 09 21:18:53.622211 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.622188 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ff4d48dfc-cnc6f"] Sep 09 21:18:53.636138 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.636095 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-cp7lf"] Sep 09 21:18:53.697499 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.697468 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-trusted-ca-bundle\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.697675 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.697505 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws77x\" (UniqueName: \"kubernetes.io/projected/cc340dd7-5dce-4186-8e38-18e1e318dc7c-kube-api-access-ws77x\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.697675 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.697554 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-config\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.697675 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.697592 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9wjk\" (UniqueName: \"kubernetes.io/projected/fbfbe2bc-0541-4fc5-9b46-aee61a222398-kube-api-access-n9wjk\") pod \"downloads-85d98c998f-cp7lf\" (UID: \"fbfbe2bc-0541-4fc5-9b46-aee61a222398\") " pod="openshift-console/downloads-85d98c998f-cp7lf" Sep 09 21:18:53.697675 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.697618 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-service-ca\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.697675 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.697637 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d111e56f-d754-4ec9-b730-09c3d470d4e4-tmp\") pod \"klusterlet-addon-workmgr-5b4c7c967d-rwvpz\" (UID: \"d111e56f-d754-4ec9-b730-09c3d470d4e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:18:53.697908 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.697687 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-serving-cert\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.697908 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.697727 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-oauth-serving-cert\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.697908 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.697767 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9wg5\" (UniqueName: \"kubernetes.io/projected/d111e56f-d754-4ec9-b730-09c3d470d4e4-kube-api-access-k9wg5\") pod \"klusterlet-addon-workmgr-5b4c7c967d-rwvpz\" (UID: \"d111e56f-d754-4ec9-b730-09c3d470d4e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:18:53.697908 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.697799 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-oauth-config\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.697908 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.697816 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d111e56f-d754-4ec9-b730-09c3d470d4e4-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b4c7c967d-rwvpz\" (UID: \"d111e56f-d754-4ec9-b730-09c3d470d4e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:18:53.699750 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.699729 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t"] Sep 09 21:18:53.702878 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.702861 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t" Sep 09 21:18:53.706282 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.706263 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f45b7\"" Sep 09 21:18:53.707047 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.707031 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Sep 09 21:18:53.720731 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.720703 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t"] Sep 09 21:18:53.798346 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798311 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9wjk\" (UniqueName: \"kubernetes.io/projected/fbfbe2bc-0541-4fc5-9b46-aee61a222398-kube-api-access-n9wjk\") pod \"downloads-85d98c998f-cp7lf\" (UID: \"fbfbe2bc-0541-4fc5-9b46-aee61a222398\") " pod="openshift-console/downloads-85d98c998f-cp7lf" Sep 09 21:18:53.798525 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798362 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-service-ca\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.798525 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798387 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d111e56f-d754-4ec9-b730-09c3d470d4e4-tmp\") pod \"klusterlet-addon-workmgr-5b4c7c967d-rwvpz\" (UID: \"d111e56f-d754-4ec9-b730-09c3d470d4e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:18:53.798525 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798407 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-serving-cert\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.798525 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798428 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-oauth-serving-cert\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.798525 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798447 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9wg5\" (UniqueName: \"kubernetes.io/projected/d111e56f-d754-4ec9-b730-09c3d470d4e4-kube-api-access-k9wg5\") pod \"klusterlet-addon-workmgr-5b4c7c967d-rwvpz\" (UID: \"d111e56f-d754-4ec9-b730-09c3d470d4e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:18:53.798525 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798476 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-oauth-config\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.798525 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798498 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d111e56f-d754-4ec9-b730-09c3d470d4e4-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b4c7c967d-rwvpz\" (UID: \"d111e56f-d754-4ec9-b730-09c3d470d4e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:18:53.798878 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798566 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/91a28cbd-41bf-44d4-8503-2aefc141c3e3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-wvt6t\" (UID: \"91a28cbd-41bf-44d4-8503-2aefc141c3e3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t" Sep 09 21:18:53.798878 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798746 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-trusted-ca-bundle\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.798878 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798802 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws77x\" (UniqueName: \"kubernetes.io/projected/cc340dd7-5dce-4186-8e38-18e1e318dc7c-kube-api-access-ws77x\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.798878 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798836 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-config\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.799063 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.798886 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d111e56f-d754-4ec9-b730-09c3d470d4e4-tmp\") pod \"klusterlet-addon-workmgr-5b4c7c967d-rwvpz\" (UID: \"d111e56f-d754-4ec9-b730-09c3d470d4e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:18:53.799262 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.799239 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-service-ca\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.799343 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.799313 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-oauth-serving-cert\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.799809 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.799783 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-config\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.799964 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.799940 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-trusted-ca-bundle\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.801196 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.801177 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d111e56f-d754-4ec9-b730-09c3d470d4e4-klusterlet-config\") pod \"klusterlet-addon-workmgr-5b4c7c967d-rwvpz\" (UID: \"d111e56f-d754-4ec9-b730-09c3d470d4e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:18:53.801282 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.801267 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-serving-cert\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.801554 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.801540 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-oauth-config\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.811677 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.811650 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9wg5\" (UniqueName: \"kubernetes.io/projected/d111e56f-d754-4ec9-b730-09c3d470d4e4-kube-api-access-k9wg5\") pod \"klusterlet-addon-workmgr-5b4c7c967d-rwvpz\" (UID: \"d111e56f-d754-4ec9-b730-09c3d470d4e4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:18:53.812210 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.812191 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9wjk\" (UniqueName: \"kubernetes.io/projected/fbfbe2bc-0541-4fc5-9b46-aee61a222398-kube-api-access-n9wjk\") pod \"downloads-85d98c998f-cp7lf\" (UID: \"fbfbe2bc-0541-4fc5-9b46-aee61a222398\") " pod="openshift-console/downloads-85d98c998f-cp7lf" Sep 09 21:18:53.812814 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.812793 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws77x\" (UniqueName: \"kubernetes.io/projected/cc340dd7-5dce-4186-8e38-18e1e318dc7c-kube-api-access-ws77x\") pod \"console-7ff4d48dfc-cnc6f\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.887874 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.887786 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:18:53.899508 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.899477 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/91a28cbd-41bf-44d4-8503-2aefc141c3e3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-wvt6t\" (UID: \"91a28cbd-41bf-44d4-8503-2aefc141c3e3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t" Sep 09 21:18:53.901721 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.901685 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/91a28cbd-41bf-44d4-8503-2aefc141c3e3-tls-certificates\") pod \"prometheus-operator-admission-webhook-78f599db79-wvt6t\" (UID: \"91a28cbd-41bf-44d4-8503-2aefc141c3e3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t" Sep 09 21:18:53.910059 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.910033 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:18:53.917201 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:53.917176 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-cp7lf" Sep 09 21:18:54.012184 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:54.012156 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t" Sep 09 21:18:54.167525 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:54.167496 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz"] Sep 09 21:18:54.170146 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:18:54.170116 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd111e56f_d754_4ec9_b730_09c3d470d4e4.slice/crio-e156c25eb012c61916982f65abdb3f1630ea7eab9fe22c71d2fc9b6c098b9fc9 WatchSource:0}: Error finding container e156c25eb012c61916982f65abdb3f1630ea7eab9fe22c71d2fc9b6c098b9fc9: Status 404 returned error can't find the container with id e156c25eb012c61916982f65abdb3f1630ea7eab9fe22c71d2fc9b6c098b9fc9 Sep 09 21:18:54.171958 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:54.171943 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:18:54.193313 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:54.193286 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ff4d48dfc-cnc6f"] Sep 09 21:18:54.194639 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:18:54.194603 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc340dd7_5dce_4186_8e38_18e1e318dc7c.slice/crio-f388ac29e1caeb1adbba8d2596949fdc43ce5df5b4f22bf027c710fd2d9490e6 WatchSource:0}: Error finding container f388ac29e1caeb1adbba8d2596949fdc43ce5df5b4f22bf027c710fd2d9490e6: Status 404 returned error can't find the container with id f388ac29e1caeb1adbba8d2596949fdc43ce5df5b4f22bf027c710fd2d9490e6 Sep 09 21:18:54.205645 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:54.205620 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-cp7lf"] Sep 09 21:18:54.207807 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:18:54.207786 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbfbe2bc_0541_4fc5_9b46_aee61a222398.slice/crio-2ad06c872bb58e4edbe1c6e7dfe051bcab271da3464e43b1be0f72c12b54f0a3 WatchSource:0}: Error finding container 2ad06c872bb58e4edbe1c6e7dfe051bcab271da3464e43b1be0f72c12b54f0a3: Status 404 returned error can't find the container with id 2ad06c872bb58e4edbe1c6e7dfe051bcab271da3464e43b1be0f72c12b54f0a3 Sep 09 21:18:54.283270 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:54.283243 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t"] Sep 09 21:18:54.285560 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:18:54.285530 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a28cbd_41bf_44d4_8503_2aefc141c3e3.slice/crio-63d8003e90c53ea654e1f1beb7656323b2f9e9d7e54284d6ad456b5f1fd3a1fa WatchSource:0}: Error finding container 63d8003e90c53ea654e1f1beb7656323b2f9e9d7e54284d6ad456b5f1fd3a1fa: Status 404 returned error can't find the container with id 63d8003e90c53ea654e1f1beb7656323b2f9e9d7e54284d6ad456b5f1fd3a1fa Sep 09 21:18:54.642164 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:54.642123 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-cp7lf" event={"ID":"fbfbe2bc-0541-4fc5-9b46-aee61a222398","Type":"ContainerStarted","Data":"2ad06c872bb58e4edbe1c6e7dfe051bcab271da3464e43b1be0f72c12b54f0a3"} Sep 09 21:18:54.643704 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:54.643668 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t" event={"ID":"91a28cbd-41bf-44d4-8503-2aefc141c3e3","Type":"ContainerStarted","Data":"63d8003e90c53ea654e1f1beb7656323b2f9e9d7e54284d6ad456b5f1fd3a1fa"} Sep 09 21:18:54.644737 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:54.644716 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff4d48dfc-cnc6f" event={"ID":"cc340dd7-5dce-4186-8e38-18e1e318dc7c","Type":"ContainerStarted","Data":"f388ac29e1caeb1adbba8d2596949fdc43ce5df5b4f22bf027c710fd2d9490e6"} Sep 09 21:18:54.645770 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:54.645751 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" event={"ID":"d111e56f-d754-4ec9-b730-09c3d470d4e4","Type":"ContainerStarted","Data":"e156c25eb012c61916982f65abdb3f1630ea7eab9fe22c71d2fc9b6c098b9fc9"} Sep 09 21:18:56.481365 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:56.481311 2553 patch_prober.go:28] interesting pod/image-registry-94b4b97c6-jlwgh container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Sep 09 21:18:56.481870 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:56.481380 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:18:56.659398 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:56.659301 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t" event={"ID":"91a28cbd-41bf-44d4-8503-2aefc141c3e3","Type":"ContainerStarted","Data":"061913f7dc1f918301d1ee806f7bd4cac1fb0d643f0ce69b6b4b156ffabfb55c"} Sep 09 21:18:56.659638 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:56.659617 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t" Sep 09 21:18:56.666127 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:56.666085 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t" Sep 09 21:18:56.680303 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:56.680068 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-78f599db79-wvt6t" podStartSLOduration=2.234586153 podStartE2EDuration="3.680047814s" totalImagesPullingTime="1.445461661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:18:56.677959997 +0000 UTC m=+546.784703851" watchObservedRunningTime="2026-09-09 21:18:56.680047814 +0000 UTC m=+546.786791641" Sep 09 21:18:57.216616 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.216261 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-v6xj7"] Sep 09 21:18:57.220639 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.220613 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.223524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.223496 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Sep 09 21:18:57.224858 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.224834 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-h78bq\"" Sep 09 21:18:57.224977 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.224905 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Sep 09 21:18:57.225054 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.225039 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Sep 09 21:18:57.232950 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.232918 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-v6xj7"] Sep 09 21:18:57.335018 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.334971 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/243e928c-cca9-496e-b035-496219854fa0-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-v6xj7\" (UID: \"243e928c-cca9-496e-b035-496219854fa0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.335223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.335037 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhl28\" (UniqueName: \"kubernetes.io/projected/243e928c-cca9-496e-b035-496219854fa0-kube-api-access-hhl28\") pod \"prometheus-operator-6f4946db56-v6xj7\" (UID: \"243e928c-cca9-496e-b035-496219854fa0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.335223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.335190 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/243e928c-cca9-496e-b035-496219854fa0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-v6xj7\" (UID: \"243e928c-cca9-496e-b035-496219854fa0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.335349 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.335261 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/243e928c-cca9-496e-b035-496219854fa0-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-v6xj7\" (UID: \"243e928c-cca9-496e-b035-496219854fa0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.436658 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.435979 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhl28\" (UniqueName: \"kubernetes.io/projected/243e928c-cca9-496e-b035-496219854fa0-kube-api-access-hhl28\") pod \"prometheus-operator-6f4946db56-v6xj7\" (UID: \"243e928c-cca9-496e-b035-496219854fa0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.436658 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.436079 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/243e928c-cca9-496e-b035-496219854fa0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-v6xj7\" (UID: \"243e928c-cca9-496e-b035-496219854fa0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.436658 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.436132 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/243e928c-cca9-496e-b035-496219854fa0-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-v6xj7\" (UID: \"243e928c-cca9-496e-b035-496219854fa0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.436658 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.436200 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/243e928c-cca9-496e-b035-496219854fa0-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-v6xj7\" (UID: \"243e928c-cca9-496e-b035-496219854fa0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.437627 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.437577 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/243e928c-cca9-496e-b035-496219854fa0-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-v6xj7\" (UID: \"243e928c-cca9-496e-b035-496219854fa0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.439547 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.439522 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/243e928c-cca9-496e-b035-496219854fa0-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-v6xj7\" (UID: \"243e928c-cca9-496e-b035-496219854fa0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.440325 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.440296 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/243e928c-cca9-496e-b035-496219854fa0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-v6xj7\" (UID: \"243e928c-cca9-496e-b035-496219854fa0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.445440 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.445396 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhl28\" (UniqueName: \"kubernetes.io/projected/243e928c-cca9-496e-b035-496219854fa0-kube-api-access-hhl28\") pod \"prometheus-operator-6f4946db56-v6xj7\" (UID: \"243e928c-cca9-496e-b035-496219854fa0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:18:57.536931 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:18:57.536828 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" Sep 09 21:19:00.034694 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:00.034666 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-v6xj7"] Sep 09 21:19:00.037339 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:19:00.037304 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod243e928c_cca9_496e_b035_496219854fa0.slice/crio-a9e341a94b17fbf7286c6e98d945b419087529a8ed0386419c6bdd58149f4a5d WatchSource:0}: Error finding container a9e341a94b17fbf7286c6e98d945b419087529a8ed0386419c6bdd58149f4a5d: Status 404 returned error can't find the container with id a9e341a94b17fbf7286c6e98d945b419087529a8ed0386419c6bdd58149f4a5d Sep 09 21:19:00.681340 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:00.681302 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff4d48dfc-cnc6f" event={"ID":"cc340dd7-5dce-4186-8e38-18e1e318dc7c","Type":"ContainerStarted","Data":"636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9"} Sep 09 21:19:00.683701 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:00.683331 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" event={"ID":"d111e56f-d754-4ec9-b730-09c3d470d4e4","Type":"ContainerStarted","Data":"9103cc04a00c888508d0de0f72e86b8cdfedc6e1d12d23971fe449a2c7b5f695"} Sep 09 21:19:00.683701 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:00.683659 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:19:00.685034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:00.685010 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" event={"ID":"243e928c-cca9-496e-b035-496219854fa0","Type":"ContainerStarted","Data":"a9e341a94b17fbf7286c6e98d945b419087529a8ed0386419c6bdd58149f4a5d"} Sep 09 21:19:00.686199 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:00.686170 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" Sep 09 21:19:00.702536 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:00.702479 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7ff4d48dfc-cnc6f" podStartSLOduration=2.138707365 podStartE2EDuration="7.702459251s" totalImagesPullingTime="5.563751886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:00.701850202 +0000 UTC m=+550.808594046" watchObservedRunningTime="2026-09-09 21:19:00.702459251 +0000 UTC m=+550.809203079" Sep 09 21:19:00.720958 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:00.720897 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5b4c7c967d-rwvpz" podStartSLOduration=2.111375713 podStartE2EDuration="7.720876479s" totalImagesPullingTime="5.609500766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:00.719159855 +0000 UTC m=+550.825903718" watchObservedRunningTime="2026-09-09 21:19:00.720876479 +0000 UTC m=+550.827620307" Sep 09 21:19:01.691738 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:01.691593 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" event={"ID":"243e928c-cca9-496e-b035-496219854fa0","Type":"ContainerStarted","Data":"3c726d865d0d9c2863816cf3efb74b9ddddd2678dbef15d0badc23fde2dd43ac"} Sep 09 21:19:01.691738 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:01.691693 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" event={"ID":"243e928c-cca9-496e-b035-496219854fa0","Type":"ContainerStarted","Data":"9595d47ce4f0d5eac9a0464a76bc54d712468123b0a17d12bedddc332e8138a8"} Sep 09 21:19:01.735498 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:01.735440 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-v6xj7" podStartSLOduration=3.443318493 podStartE2EDuration="4.735416859s" totalImagesPullingTime="1.292098366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:01.734766271 +0000 UTC m=+551.841510122" watchObservedRunningTime="2026-09-09 21:19:01.735416859 +0000 UTC m=+551.842160687" Sep 09 21:19:03.573630 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.573598 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:19:03.655747 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.655710 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6"] Sep 09 21:19:03.659654 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.659623 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.663931 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.663905 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Sep 09 21:19:03.664456 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.664225 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Sep 09 21:19:03.664456 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.664255 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-q974g\"" Sep 09 21:19:03.664456 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.664384 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Sep 09 21:19:03.681474 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.681439 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6"] Sep 09 21:19:03.690427 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.690398 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-p9lg4"] Sep 09 21:19:03.694456 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.694424 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.698160 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.698131 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Sep 09 21:19:03.698680 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.698650 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Sep 09 21:19:03.698941 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.698920 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bz8x8\"" Sep 09 21:19:03.700350 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.700316 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Sep 09 21:19:03.798248 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798209 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dttl\" (UniqueName: \"kubernetes.io/projected/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-api-access-2dttl\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.798497 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798256 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-tls\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.798497 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798374 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/26d8967d-d973-40cb-b7a8-fa88046c521a-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.798497 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798422 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.798497 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798453 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.798683 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798519 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.798683 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798554 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.798683 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798583 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-wtmp\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.798683 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798611 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-accelerators-collector-config\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.798683 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798669 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-sys\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.798857 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798697 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-root\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.798857 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798723 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmqc7\" (UniqueName: \"kubernetes.io/projected/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-kube-api-access-dmqc7\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.798857 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798762 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-textfile\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.798857 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798787 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-metrics-client-ca\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.798857 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.798818 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26d8967d-d973-40cb-b7a8-fa88046c521a-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.899350 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899260 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.899350 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899304 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.899584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899353 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.899584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899382 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.899584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899405 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-wtmp\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.899584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899431 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-accelerators-collector-config\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.899584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899454 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-sys\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.899584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899475 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-root\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.899584 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:19:03.899516 2553 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Sep 09 21:19:03.899584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899564 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-root\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.899584 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:19:03.899587 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-state-metrics-tls podName:26d8967d-d973-40cb-b7a8-fa88046c521a nodeName:}" failed. No retries permitted until 2026-09-09 21:19:04.399568747 +0000 UTC m=+554.506312573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-fwmw6" (UID: "26d8967d-d973-40cb-b7a8-fa88046c521a") : secret "kube-state-metrics-tls" not found Sep 09 21:19:03.900010 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899609 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-sys\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.900010 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899661 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqc7\" (UniqueName: \"kubernetes.io/projected/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-kube-api-access-dmqc7\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.900010 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899692 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-wtmp\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.900010 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899694 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-textfile\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.900010 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899738 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-metrics-client-ca\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.900010 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899771 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26d8967d-d973-40cb-b7a8-fa88046c521a-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.900010 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899812 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dttl\" (UniqueName: \"kubernetes.io/projected/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-api-access-2dttl\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.900010 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899842 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-tls\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.900010 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.899903 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/26d8967d-d973-40cb-b7a8-fa88046c521a-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.900010 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.900004 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-textfile\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.900569 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.900320 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-metrics-client-ca\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.900905 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.900702 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.900905 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.900873 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-accelerators-collector-config\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.900905 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.900869 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26d8967d-d973-40cb-b7a8-fa88046c521a-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.901068 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.900879 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/26d8967d-d973-40cb-b7a8-fa88046c521a-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.902534 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.902508 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.902639 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.902579 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.903113 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.903074 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-node-exporter-tls\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.909637 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.909607 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmqc7\" (UniqueName: \"kubernetes.io/projected/b1c6e518-b7ac-4e11-bc64-f67e8d0eddae-kube-api-access-dmqc7\") pod \"node-exporter-p9lg4\" (UID: \"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae\") " pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:03.909944 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.909924 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dttl\" (UniqueName: \"kubernetes.io/projected/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-api-access-2dttl\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:03.910726 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.910479 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:19:03.910726 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.910543 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:19:03.917722 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:03.917695 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:19:04.008198 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:04.008162 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-p9lg4" Sep 09 21:19:04.021875 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:19:04.021840 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1c6e518_b7ac_4e11_bc64_f67e8d0eddae.slice/crio-4675e1b0f8355732196476baa5356ca9796fe1e688fdcc1ebe34955a9081f57d WatchSource:0}: Error finding container 4675e1b0f8355732196476baa5356ca9796fe1e688fdcc1ebe34955a9081f57d: Status 404 returned error can't find the container with id 4675e1b0f8355732196476baa5356ca9796fe1e688fdcc1ebe34955a9081f57d Sep 09 21:19:04.404791 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:04.404758 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:04.407507 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:04.407482 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/26d8967d-d973-40cb-b7a8-fa88046c521a-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-fwmw6\" (UID: \"26d8967d-d973-40cb-b7a8-fa88046c521a\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:04.573039 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:04.573003 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" Sep 09 21:19:04.715896 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:04.715605 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p9lg4" event={"ID":"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae","Type":"ContainerStarted","Data":"4675e1b0f8355732196476baa5356ca9796fe1e688fdcc1ebe34955a9081f57d"} Sep 09 21:19:04.721291 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:04.721207 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:19:04.871153 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:04.871072 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6"] Sep 09 21:19:05.609194 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.609160 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv"] Sep 09 21:19:05.613134 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.613110 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.615793 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.615771 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Sep 09 21:19:05.615928 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.615874 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Sep 09 21:19:05.616234 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.616172 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Sep 09 21:19:05.616407 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.616359 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lv4ws\"" Sep 09 21:19:05.616960 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.616943 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Sep 09 21:19:05.617272 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.616999 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-3u7hhb76dl6mm\"" Sep 09 21:19:05.617438 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.617001 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Sep 09 21:19:05.622439 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.622414 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv"] Sep 09 21:19:05.717431 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.717401 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.717891 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.717440 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55p58\" (UniqueName: \"kubernetes.io/projected/be52978f-2862-4b78-ac25-e85d314114da-kube-api-access-55p58\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.717891 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.717474 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-tls\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.717891 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.717508 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.717891 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.717537 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.717891 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.717598 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.717891 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.717630 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be52978f-2862-4b78-ac25-e85d314114da-metrics-client-ca\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.717891 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.717714 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-grpc-tls\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.818242 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.818201 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-grpc-tls\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.818449 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.818279 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.818449 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.818310 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55p58\" (UniqueName: \"kubernetes.io/projected/be52978f-2862-4b78-ac25-e85d314114da-kube-api-access-55p58\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.818449 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.818382 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-tls\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.818449 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.818410 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.818449 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.818439 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.818641 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.818506 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.819461 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.819154 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be52978f-2862-4b78-ac25-e85d314114da-metrics-client-ca\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.820027 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.819976 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be52978f-2862-4b78-ac25-e85d314114da-metrics-client-ca\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.821595 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.821447 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.822014 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.821783 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-tls\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.822014 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.821977 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.822188 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.822156 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.822267 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.822242 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.822528 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.822505 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/be52978f-2862-4b78-ac25-e85d314114da-secret-grpc-tls\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.828023 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.827993 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55p58\" (UniqueName: \"kubernetes.io/projected/be52978f-2862-4b78-ac25-e85d314114da-kube-api-access-55p58\") pod \"thanos-querier-6ddd999fd7-tsjkv\" (UID: \"be52978f-2862-4b78-ac25-e85d314114da\") " pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:05.925690 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:05.925656 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:08.391329 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.391300 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v"] Sep 09 21:19:08.395026 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.395006 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" Sep 09 21:19:08.397717 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.397693 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Sep 09 21:19:08.397825 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.397693 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-m95hf\"" Sep 09 21:19:08.406280 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.406253 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v"] Sep 09 21:19:08.546222 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.546190 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ab209ce4-7847-40aa-886c-e4e5aaf19f1a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-pzl2v\" (UID: \"ab209ce4-7847-40aa-886c-e4e5aaf19f1a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" Sep 09 21:19:08.546398 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.546358 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-248jb\" (UniqueName: \"kubernetes.io/projected/ab209ce4-7847-40aa-886c-e4e5aaf19f1a-kube-api-access-248jb\") pod \"monitoring-plugin-85bbbbbf4-pzl2v\" (UID: \"ab209ce4-7847-40aa-886c-e4e5aaf19f1a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" Sep 09 21:19:08.647788 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.647556 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-248jb\" (UniqueName: \"kubernetes.io/projected/ab209ce4-7847-40aa-886c-e4e5aaf19f1a-kube-api-access-248jb\") pod \"monitoring-plugin-85bbbbbf4-pzl2v\" (UID: \"ab209ce4-7847-40aa-886c-e4e5aaf19f1a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" Sep 09 21:19:08.648554 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.647904 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ab209ce4-7847-40aa-886c-e4e5aaf19f1a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-pzl2v\" (UID: \"ab209ce4-7847-40aa-886c-e4e5aaf19f1a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" Sep 09 21:19:08.648554 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:19:08.648210 2553 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Sep 09 21:19:08.648554 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:19:08.648286 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab209ce4-7847-40aa-886c-e4e5aaf19f1a-monitoring-plugin-cert podName:ab209ce4-7847-40aa-886c-e4e5aaf19f1a nodeName:}" failed. No retries permitted until 2026-09-09 21:19:09.14826702 +0000 UTC m=+559.255010839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/ab209ce4-7847-40aa-886c-e4e5aaf19f1a-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-pzl2v" (UID: "ab209ce4-7847-40aa-886c-e4e5aaf19f1a") : secret "monitoring-plugin-cert" not found Sep 09 21:19:08.657084 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.657054 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-248jb\" (UniqueName: \"kubernetes.io/projected/ab209ce4-7847-40aa-886c-e4e5aaf19f1a-kube-api-access-248jb\") pod \"monitoring-plugin-85bbbbbf4-pzl2v\" (UID: \"ab209ce4-7847-40aa-886c-e4e5aaf19f1a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" Sep 09 21:19:08.812140 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.811970 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86b4b89844-vklwg"] Sep 09 21:19:08.817598 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.817572 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:08.835039 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.835005 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86b4b89844-vklwg"] Sep 09 21:19:08.922828 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.922658 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-84688c96dd-bwqdl"] Sep 09 21:19:08.926821 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.926797 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:08.929686 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.929660 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Sep 09 21:19:08.929827 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.929754 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Sep 09 21:19:08.929898 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.929883 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-c6rh7\"" Sep 09 21:19:08.930037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.929998 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Sep 09 21:19:08.930037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.929998 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Sep 09 21:19:08.930349 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.930313 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Sep 09 21:19:08.935737 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.935714 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Sep 09 21:19:08.940411 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.940386 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-84688c96dd-bwqdl"] Sep 09 21:19:08.950393 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.950359 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-serving-cert\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:08.950550 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.950427 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzkzd\" (UniqueName: \"kubernetes.io/projected/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-kube-api-access-bzkzd\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:08.950550 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.950466 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-oauth-serving-cert\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:08.950550 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.950508 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-config\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:08.950550 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.950535 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-trusted-ca-bundle\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:08.950757 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.950573 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-service-ca\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:08.950757 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:08.950650 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-oauth-config\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.051809 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.051752 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.051995 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.051815 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-serving-cert\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.051995 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.051893 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.051995 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.051939 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-metrics-client-ca\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.051995 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.051978 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzkzd\" (UniqueName: \"kubernetes.io/projected/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-kube-api-access-bzkzd\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.052235 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.052004 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-serving-certs-ca-bundle\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.052235 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.052167 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-federate-client-tls\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.052235 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.052215 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-oauth-serving-cert\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.052373 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.052247 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsthf\" (UniqueName: \"kubernetes.io/projected/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-kube-api-access-bsthf\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.052373 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.052289 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-secret-telemeter-client\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.052373 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.052335 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-config\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.052523 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.052384 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-trusted-ca-bundle\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.052523 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.052464 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-service-ca\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.052523 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.052494 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-telemeter-client-tls\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.052685 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.052558 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-oauth-config\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.052966 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.052944 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-oauth-serving-cert\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.053083 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.053024 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-config\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.053309 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.053281 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-service-ca\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.053468 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.053445 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-trusted-ca-bundle\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.054788 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.054747 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-serving-cert\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.055374 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.055334 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-oauth-config\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.061759 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.061735 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzkzd\" (UniqueName: \"kubernetes.io/projected/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-kube-api-access-bzkzd\") pod \"console-86b4b89844-vklwg\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.130147 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.130077 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:09.153777 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.153741 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-telemeter-client-tls\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.153966 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.153803 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.153966 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.153844 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.153966 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.153871 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-metrics-client-ca\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.153966 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.153903 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-serving-certs-ca-bundle\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.154414 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.154382 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-federate-client-tls\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.154537 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.154438 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsthf\" (UniqueName: \"kubernetes.io/projected/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-kube-api-access-bsthf\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.154790 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.154696 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-secret-telemeter-client\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.154790 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.154777 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-serving-certs-ca-bundle\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.154926 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.154810 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ab209ce4-7847-40aa-886c-e4e5aaf19f1a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-pzl2v\" (UID: \"ab209ce4-7847-40aa-886c-e4e5aaf19f1a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" Sep 09 21:19:09.155366 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.155315 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.155577 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.155534 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-metrics-client-ca\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.156914 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.156886 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.157017 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.156971 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-telemeter-client-tls\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.157742 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.157703 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-federate-client-tls\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.157839 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.157766 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ab209ce4-7847-40aa-886c-e4e5aaf19f1a-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-pzl2v\" (UID: \"ab209ce4-7847-40aa-886c-e4e5aaf19f1a\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" Sep 09 21:19:09.158076 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.158057 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-secret-telemeter-client\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.162933 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.162913 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsthf\" (UniqueName: \"kubernetes.io/projected/0ffa260b-9f12-44e6-823d-a4ed52e2bafa-kube-api-access-bsthf\") pod \"telemeter-client-84688c96dd-bwqdl\" (UID: \"0ffa260b-9f12-44e6-823d-a4ed52e2bafa\") " pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.238862 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.238826 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" Sep 09 21:19:09.307526 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.307486 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" Sep 09 21:19:09.838325 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.838262 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 21:19:09.843308 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.843283 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.847475 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.847317 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Sep 09 21:19:09.847475 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.847335 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Sep 09 21:19:09.847475 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.847335 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9snlx\"" Sep 09 21:19:09.847475 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.847364 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Sep 09 21:19:09.847743 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.847656 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Sep 09 21:19:09.848368 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.848329 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Sep 09 21:19:09.848368 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.848364 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Sep 09 21:19:09.848546 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.848402 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Sep 09 21:19:09.848546 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.848434 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Sep 09 21:19:09.848546 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.848445 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Sep 09 21:19:09.848705 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.848567 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Sep 09 21:19:09.848761 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.848707 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Sep 09 21:19:09.848811 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.848777 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8ctkahe76k1fq\"" Sep 09 21:19:09.849177 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.849160 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Sep 09 21:19:09.849669 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.849614 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Sep 09 21:19:09.851376 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.851356 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Sep 09 21:19:09.854367 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.854191 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Sep 09 21:19:09.859581 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.859539 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 21:19:09.863348 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863319 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863483 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863359 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863483 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863385 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863483 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863430 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863483 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863458 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863699 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863531 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-config\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863699 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863586 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863699 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863624 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863699 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863648 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-web-config\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863699 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863679 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863942 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863714 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/527baa92-bd10-4f5d-a15c-b68cde8e0642-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863942 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863748 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863942 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863800 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863942 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863842 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863942 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863868 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.863942 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863909 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfd5x\" (UniqueName: \"kubernetes.io/projected/527baa92-bd10-4f5d-a15c-b68cde8e0642-kube-api-access-vfd5x\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.864243 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863954 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.864243 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.863981 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/527baa92-bd10-4f5d-a15c-b68cde8e0642-config-out\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.864243 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.864003 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.864243 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.864024 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.965479 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965443 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.965683 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965499 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.965683 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965626 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.965683 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965670 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfd5x\" (UniqueName: \"kubernetes.io/projected/527baa92-bd10-4f5d-a15c-b68cde8e0642-kube-api-access-vfd5x\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.965868 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965710 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.965868 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965745 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/527baa92-bd10-4f5d-a15c-b68cde8e0642-config-out\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.965868 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965770 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.965868 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965796 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.965868 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965863 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966250 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965888 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966250 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965927 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966250 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965953 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966250 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.965980 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966250 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.966026 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-config\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966250 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.966092 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966250 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.966144 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966250 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.966179 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-web-config\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966250 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.966207 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966250 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.966246 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/527baa92-bd10-4f5d-a15c-b68cde8e0642-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966718 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.966277 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966718 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.966707 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.966821 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.966773 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.967814 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.967473 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.968769 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.968745 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.969451 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.969279 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.970896 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.970808 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.971009 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.970968 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-web-config\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.973507 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.971305 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.973507 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.971499 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-config\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.973507 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.971775 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.973507 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.972159 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.973507 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.972488 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.973507 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.972610 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.973507 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.973407 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.973982 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.973585 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/527baa92-bd10-4f5d-a15c-b68cde8e0642-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.973982 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.973851 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.974580 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.974227 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/527baa92-bd10-4f5d-a15c-b68cde8e0642-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.974580 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.974315 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/527baa92-bd10-4f5d-a15c-b68cde8e0642-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.974580 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.974426 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfd5x\" (UniqueName: \"kubernetes.io/projected/527baa92-bd10-4f5d-a15c-b68cde8e0642-kube-api-access-vfd5x\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:09.975151 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:09.975131 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/527baa92-bd10-4f5d-a15c-b68cde8e0642-config-out\") pod \"prometheus-k8s-0\" (UID: \"527baa92-bd10-4f5d-a15c-b68cde8e0642\") " pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:10.158803 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:10.158720 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:11.989433 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:19:11.989399 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26d8967d_d973_40cb_b7a8_fa88046c521a.slice/crio-4f3ec880b7a7b858cfb46c18fd84fe43c80a9c77887222a37d64a0481a932fce WatchSource:0}: Error finding container 4f3ec880b7a7b858cfb46c18fd84fe43c80a9c77887222a37d64a0481a932fce: Status 404 returned error can't find the container with id 4f3ec880b7a7b858cfb46c18fd84fe43c80a9c77887222a37d64a0481a932fce Sep 09 21:19:12.562612 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.562575 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86b4b89844-vklwg"] Sep 09 21:19:12.575693 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.575070 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Sep 09 21:19:12.577801 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:19:12.577754 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod527baa92_bd10_4f5d_a15c_b68cde8e0642.slice/crio-4c543496729e76f924a917d50e7f27b1a0352870ea51ebfaa8a093adfaa55fa9 WatchSource:0}: Error finding container 4c543496729e76f924a917d50e7f27b1a0352870ea51ebfaa8a093adfaa55fa9: Status 404 returned error can't find the container with id 4c543496729e76f924a917d50e7f27b1a0352870ea51ebfaa8a093adfaa55fa9 Sep 09 21:19:12.581548 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.581460 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-84688c96dd-bwqdl"] Sep 09 21:19:12.585887 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:19:12.585850 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ffa260b_9f12_44e6_823d_a4ed52e2bafa.slice/crio-b7202cb0ef96b73d4e3619b5c1b4136e458b1aa0e1271e700075ee6a5c654bdd WatchSource:0}: Error finding container b7202cb0ef96b73d4e3619b5c1b4136e458b1aa0e1271e700075ee6a5c654bdd: Status 404 returned error can't find the container with id b7202cb0ef96b73d4e3619b5c1b4136e458b1aa0e1271e700075ee6a5c654bdd Sep 09 21:19:12.597462 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.597402 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v"] Sep 09 21:19:12.597836 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:19:12.597808 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab209ce4_7847_40aa_886c_e4e5aaf19f1a.slice/crio-80cad75af5c8d30869e2141ff2800813849371461de7d61cd20b7ac1d88f6c27 WatchSource:0}: Error finding container 80cad75af5c8d30869e2141ff2800813849371461de7d61cd20b7ac1d88f6c27: Status 404 returned error can't find the container with id 80cad75af5c8d30869e2141ff2800813849371461de7d61cd20b7ac1d88f6c27 Sep 09 21:19:12.598284 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.598258 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv"] Sep 09 21:19:12.756606 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.756489 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" event={"ID":"be52978f-2862-4b78-ac25-e85d314114da","Type":"ContainerStarted","Data":"c4b2fb2da94a0b76259fc48b66fa52cc7aa95ca123255a2c188a67187cda3b1b"} Sep 09 21:19:12.758334 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.758305 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"527baa92-bd10-4f5d-a15c-b68cde8e0642","Type":"ContainerStarted","Data":"4c543496729e76f924a917d50e7f27b1a0352870ea51ebfaa8a093adfaa55fa9"} Sep 09 21:19:12.760813 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.760782 2553 generic.go:356] "Generic (PLEG): container finished" podID="b1c6e518-b7ac-4e11-bc64-f67e8d0eddae" containerID="2e48a88aacc0a73b0041e7007c477f2677bb5b3a020d1b53492cb8a747a40043" exitCode=0 Sep 09 21:19:12.760924 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.760881 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p9lg4" event={"ID":"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae","Type":"ContainerDied","Data":"2e48a88aacc0a73b0041e7007c477f2677bb5b3a020d1b53492cb8a747a40043"} Sep 09 21:19:12.763871 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.763797 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b4b89844-vklwg" event={"ID":"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a","Type":"ContainerStarted","Data":"80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6"} Sep 09 21:19:12.763871 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.763842 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b4b89844-vklwg" event={"ID":"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a","Type":"ContainerStarted","Data":"4681092c358c3c7badf7bb44aa09d27331eb6b581dd3ef5635680f1366255396"} Sep 09 21:19:12.767791 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.767728 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" event={"ID":"0ffa260b-9f12-44e6-823d-a4ed52e2bafa","Type":"ContainerStarted","Data":"b7202cb0ef96b73d4e3619b5c1b4136e458b1aa0e1271e700075ee6a5c654bdd"} Sep 09 21:19:12.769925 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.769858 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" event={"ID":"26d8967d-d973-40cb-b7a8-fa88046c521a","Type":"ContainerStarted","Data":"4f3ec880b7a7b858cfb46c18fd84fe43c80a9c77887222a37d64a0481a932fce"} Sep 09 21:19:12.772456 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.771916 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-cp7lf" event={"ID":"fbfbe2bc-0541-4fc5-9b46-aee61a222398","Type":"ContainerStarted","Data":"95e1690777f4d223b8bb1ee9b79f2d361a1d8eddb60e258bf1db232fa6308b2f"} Sep 09 21:19:12.772456 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.772380 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-cp7lf" Sep 09 21:19:12.775526 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.775495 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" event={"ID":"ab209ce4-7847-40aa-886c-e4e5aaf19f1a","Type":"ContainerStarted","Data":"80cad75af5c8d30869e2141ff2800813849371461de7d61cd20b7ac1d88f6c27"} Sep 09 21:19:12.780455 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.780428 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-cp7lf" Sep 09 21:19:12.801174 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.801119 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86b4b89844-vklwg" podStartSLOduration=4.801078413 podStartE2EDuration="4.801078413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:12.800725613 +0000 UTC m=+562.907469459" watchObservedRunningTime="2026-09-09 21:19:12.801078413 +0000 UTC m=+562.907822259" Sep 09 21:19:12.821878 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:12.821770 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-cp7lf" podStartSLOduration=1.71031289 podStartE2EDuration="19.821752123s" totalImagesPullingTime="18.111439233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:12.821170407 +0000 UTC m=+562.927914248" watchObservedRunningTime="2026-09-09 21:19:12.821752123 +0000 UTC m=+562.928495951" Sep 09 21:19:13.810134 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:13.809720 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p9lg4" event={"ID":"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae","Type":"ContainerStarted","Data":"695e43e90c48c1c1687f55829a287f8e1bcd3949ac496f03eeae4100de9cdb58"} Sep 09 21:19:13.810134 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:13.809848 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-p9lg4" event={"ID":"b1c6e518-b7ac-4e11-bc64-f67e8d0eddae","Type":"ContainerStarted","Data":"59edd3d5e44c7539f215b35511854db9e4c9106eab9a6cd0ef291fdae5d68a8d"} Sep 09 21:19:13.822503 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:13.822466 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" event={"ID":"26d8967d-d973-40cb-b7a8-fa88046c521a","Type":"ContainerStarted","Data":"b02368581519641b525fb45feb4c2339ee13ccbfcbc091017ee57e566c0d4eb0"} Sep 09 21:19:13.823802 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:13.823778 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" event={"ID":"26d8967d-d973-40cb-b7a8-fa88046c521a","Type":"ContainerStarted","Data":"ed285bfad278d1ed6bb30c06011da0a4499809d6acc052bd0901f78da447a8f9"} Sep 09 21:19:13.841879 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:13.841827 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-p9lg4" podStartSLOduration=2.5192132799999998 podStartE2EDuration="10.841801677s" totalImagesPullingTime="8.24757271s" totalInitContainerRuntime="75.015687ms" isStatefulPod=true podCreationTimestamp="2026-09-09 21:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:13.838679128 +0000 UTC m=+563.945422973" watchObservedRunningTime="2026-09-09 21:19:13.841801677 +0000 UTC m=+563.948545503" Sep 09 21:19:14.843406 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:14.842915 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" event={"ID":"26d8967d-d973-40cb-b7a8-fa88046c521a","Type":"ContainerStarted","Data":"4ccbf8769230fd647df50ad1a74e50ebecfe619d279c04f47f79398def448abd"} Sep 09 21:19:14.864651 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:14.862562 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-fwmw6" podStartSLOduration=10.525244645 podStartE2EDuration="11.862537723s" totalImagesPullingTime="1.337293078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:14.861671817 +0000 UTC m=+564.968415660" watchObservedRunningTime="2026-09-09 21:19:14.862537723 +0000 UTC m=+564.969281549" Sep 09 21:19:16.801640 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:16.801602 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-94b4b97c6-jlwgh"] Sep 09 21:19:18.865588 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.865483 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" event={"ID":"ab209ce4-7847-40aa-886c-e4e5aaf19f1a","Type":"ContainerStarted","Data":"bf1b611fc0b96834afd209f82fb104c9d96cbf487b4bc54f34edc88b94926262"} Sep 09 21:19:18.866146 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.865672 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" Sep 09 21:19:18.868007 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.867977 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" event={"ID":"be52978f-2862-4b78-ac25-e85d314114da","Type":"ContainerStarted","Data":"94c05cb4a080cad9437189cb77b4263e26d308eb00c66d526c420c834132a4d8"} Sep 09 21:19:18.868146 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.868047 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" event={"ID":"be52978f-2862-4b78-ac25-e85d314114da","Type":"ContainerStarted","Data":"b62e6175b7825ef36d5bd65d2ca605996c027b4f8a611ffa409f6dcd0087ecfb"} Sep 09 21:19:18.868146 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.868080 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" event={"ID":"be52978f-2862-4b78-ac25-e85d314114da","Type":"ContainerStarted","Data":"969625fa256aca42a46008e733e6409220433d2890b03301acb73da8caa04866"} Sep 09 21:19:18.869945 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.869915 2553 generic.go:356] "Generic (PLEG): container finished" podID="527baa92-bd10-4f5d-a15c-b68cde8e0642" containerID="a05b743aeeb9c7b67bdc859dccfda4d72d5a3a8b8579432f5b057e9893e53baf" exitCode=0 Sep 09 21:19:18.870058 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.869955 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"527baa92-bd10-4f5d-a15c-b68cde8e0642","Type":"ContainerDied","Data":"a05b743aeeb9c7b67bdc859dccfda4d72d5a3a8b8579432f5b057e9893e53baf"} Sep 09 21:19:18.871791 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.871748 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" Sep 09 21:19:18.872943 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.872920 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" event={"ID":"0ffa260b-9f12-44e6-823d-a4ed52e2bafa","Type":"ContainerStarted","Data":"a19474fbdfbd1af9461e5fad067ea928dacc2b5339f59755c0c37cb58f2b8851"} Sep 09 21:19:18.873032 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.872973 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" event={"ID":"0ffa260b-9f12-44e6-823d-a4ed52e2bafa","Type":"ContainerStarted","Data":"9fb553d0f3d4dce27dc9adf112f33ab63ab9bb218dd08d3cd94062ddb0e59669"} Sep 09 21:19:18.873032 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.873008 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" event={"ID":"0ffa260b-9f12-44e6-823d-a4ed52e2bafa","Type":"ContainerStarted","Data":"64df6d27be47e35d36946c546fb7502e15f7517daf421b14fecb03878f5648ca"} Sep 09 21:19:18.885501 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.885448 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-pzl2v" podStartSLOduration=5.427167894 podStartE2EDuration="10.88543005s" totalImagesPullingTime="5.458262156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:19:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:18.884536778 +0000 UTC m=+568.991280626" watchObservedRunningTime="2026-09-09 21:19:18.88543005 +0000 UTC m=+568.992173876" Sep 09 21:19:18.910747 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:18.910699 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-84688c96dd-bwqdl" podStartSLOduration=5.436010104 podStartE2EDuration="10.910683251s" totalImagesPullingTime="5.474673147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:19:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:18.90800902 +0000 UTC m=+569.014752869" watchObservedRunningTime="2026-09-09 21:19:18.910683251 +0000 UTC m=+569.017427077" Sep 09 21:19:19.000526 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:19.000487 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86b4b89844-vklwg"] Sep 09 21:19:19.130970 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:19.130870 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:20.901696 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:20.901653 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" event={"ID":"be52978f-2862-4b78-ac25-e85d314114da","Type":"ContainerStarted","Data":"c63a59ec786fd02d27d6c63915ccd5c01ae2adc6eaf2a7daa480990f1025de4a"} Sep 09 21:19:20.902209 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:20.901776 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" event={"ID":"be52978f-2862-4b78-ac25-e85d314114da","Type":"ContainerStarted","Data":"06767a61b5dd40d78f3e7c6d0149275c70b5fead45d4df413604f53c3107a599"} Sep 09 21:19:20.902209 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:20.901823 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" event={"ID":"be52978f-2862-4b78-ac25-e85d314114da","Type":"ContainerStarted","Data":"72e670c27f937d62ac8c6790628666e9e5b1d14063bd3a2efc322eabab0c59b2"} Sep 09 21:19:20.926674 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:20.926518 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" podStartSLOduration=9.071245203 podStartE2EDuration="15.926496744s" totalImagesPullingTime="6.855251541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:19:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:20.925254256 +0000 UTC m=+571.031998101" watchObservedRunningTime="2026-09-09 21:19:20.926496744 +0000 UTC m=+571.033240598" Sep 09 21:19:21.907114 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:21.907077 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:23.919666 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:23.919593 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"527baa92-bd10-4f5d-a15c-b68cde8e0642","Type":"ContainerStarted","Data":"c45f4abcbaf2aa604514ce12695256a0afb0ef7b13a5398608a1315ee78f5bb6"} Sep 09 21:19:23.919666 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:23.919656 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"527baa92-bd10-4f5d-a15c-b68cde8e0642","Type":"ContainerStarted","Data":"f7e9ed71aae2d860ee3c56dd2a8390a5adbed6be4b16cbbcaea07baf78841bcf"} Sep 09 21:19:23.920037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:23.919683 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"527baa92-bd10-4f5d-a15c-b68cde8e0642","Type":"ContainerStarted","Data":"883b07181b7fb73edf566e6f9f7d2b04da4eb94a1c34473b835a0ff635fd7907"} Sep 09 21:19:23.920037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:23.919708 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"527baa92-bd10-4f5d-a15c-b68cde8e0642","Type":"ContainerStarted","Data":"2a57c6740cbd9b5887a63131052181749e644f4d9bbb81df4d4614df7313b080"} Sep 09 21:19:23.920037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:23.919731 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"527baa92-bd10-4f5d-a15c-b68cde8e0642","Type":"ContainerStarted","Data":"8a1a0b1229e5a25fb6f2aa24c4e1c62c32d616cc2c2608fa1e6b80fbb9ba9d9d"} Sep 09 21:19:24.928060 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:24.928027 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"527baa92-bd10-4f5d-a15c-b68cde8e0642","Type":"ContainerStarted","Data":"0d46e1fea8186125d7bbc510d0e18260aea86c303b88183201c730303cc56cc5"} Sep 09 21:19:24.961337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:24.961264 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.181437647 podStartE2EDuration="15.961246997s" totalImagesPullingTime="9.742302787s" totalInitContainerRuntime="37.506563ms" isStatefulPod=false podCreationTimestamp="2026-09-09 21:19:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:19:24.958382545 +0000 UTC m=+575.065126393" watchObservedRunningTime="2026-09-09 21:19:24.961246997 +0000 UTC m=+575.067990822" Sep 09 21:19:25.159144 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:25.159091 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:19:27.917039 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:27.917008 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6ddd999fd7-tsjkv" Sep 09 21:19:29.957862 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:29.957826 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ff4d48dfc-cnc6f"] Sep 09 21:19:41.831075 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:41.830998 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" containerID="cri-o://0e85901e0972abc0f701aaf6ed77c73ece10ed44c7ca9bce0e71fee2e5493422" gracePeriod=30 Sep 09 21:19:42.012090 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.012058 2553 generic.go:356] "Generic (PLEG): container finished" podID="ece35fe8-abce-464f-a888-9798a5725a40" containerID="0e85901e0972abc0f701aaf6ed77c73ece10ed44c7ca9bce0e71fee2e5493422" exitCode=0 Sep 09 21:19:42.012258 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.012123 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerDied","Data":"0e85901e0972abc0f701aaf6ed77c73ece10ed44c7ca9bce0e71fee2e5493422"} Sep 09 21:19:42.012258 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.012172 2553 scope.go:117] "RemoveContainer" containerID="959c1e868c6d78fe7eb8e8f188c0cdc8b04d14477a9133fc7d7b3dcb375f1525" Sep 09 21:19:42.081610 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.081549 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:19:42.105707 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.105678 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ece35fe8-abce-464f-a888-9798a5725a40-image-registry-private-configuration\") pod \"ece35fe8-abce-464f-a888-9798a5725a40\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " Sep 09 21:19:42.105707 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.105711 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ece35fe8-abce-464f-a888-9798a5725a40-ca-trust-extracted\") pod \"ece35fe8-abce-464f-a888-9798a5725a40\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " Sep 09 21:19:42.105934 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.105745 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ece35fe8-abce-464f-a888-9798a5725a40-registry-certificates\") pod \"ece35fe8-abce-464f-a888-9798a5725a40\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " Sep 09 21:19:42.105934 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.105787 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ece35fe8-abce-464f-a888-9798a5725a40-installation-pull-secrets\") pod \"ece35fe8-abce-464f-a888-9798a5725a40\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " Sep 09 21:19:42.105934 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.105816 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdw9k\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-kube-api-access-pdw9k\") pod \"ece35fe8-abce-464f-a888-9798a5725a40\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " Sep 09 21:19:42.105934 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.105870 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls\") pod \"ece35fe8-abce-464f-a888-9798a5725a40\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " Sep 09 21:19:42.105934 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.105894 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-bound-sa-token\") pod \"ece35fe8-abce-464f-a888-9798a5725a40\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " Sep 09 21:19:42.105934 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.105933 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ece35fe8-abce-464f-a888-9798a5725a40-trusted-ca\") pod \"ece35fe8-abce-464f-a888-9798a5725a40\" (UID: \"ece35fe8-abce-464f-a888-9798a5725a40\") " Sep 09 21:19:42.106279 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.106255 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ece35fe8-abce-464f-a888-9798a5725a40-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ece35fe8-abce-464f-a888-9798a5725a40" (UID: "ece35fe8-abce-464f-a888-9798a5725a40"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:19:42.106628 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.106605 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ece35fe8-abce-464f-a888-9798a5725a40-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ece35fe8-abce-464f-a888-9798a5725a40" (UID: "ece35fe8-abce-464f-a888-9798a5725a40"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:19:42.108824 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.108780 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ece35fe8-abce-464f-a888-9798a5725a40-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ece35fe8-abce-464f-a888-9798a5725a40" (UID: "ece35fe8-abce-464f-a888-9798a5725a40"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:19:42.108998 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.108869 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ece35fe8-abce-464f-a888-9798a5725a40-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ece35fe8-abce-464f-a888-9798a5725a40" (UID: "ece35fe8-abce-464f-a888-9798a5725a40"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:19:42.109435 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.109333 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ece35fe8-abce-464f-a888-9798a5725a40" (UID: "ece35fe8-abce-464f-a888-9798a5725a40"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:19:42.109555 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.109530 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-kube-api-access-pdw9k" (OuterVolumeSpecName: "kube-api-access-pdw9k") pod "ece35fe8-abce-464f-a888-9798a5725a40" (UID: "ece35fe8-abce-464f-a888-9798a5725a40"). InnerVolumeSpecName "kube-api-access-pdw9k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:19:42.110619 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.110556 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ece35fe8-abce-464f-a888-9798a5725a40" (UID: "ece35fe8-abce-464f-a888-9798a5725a40"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:19:42.115573 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.115551 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ece35fe8-abce-464f-a888-9798a5725a40-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ece35fe8-abce-464f-a888-9798a5725a40" (UID: "ece35fe8-abce-464f-a888-9798a5725a40"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:19:42.206714 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.206677 2553 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ece35fe8-abce-464f-a888-9798a5725a40-image-registry-private-configuration\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:42.206714 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.206709 2553 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ece35fe8-abce-464f-a888-9798a5725a40-ca-trust-extracted\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:42.206714 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.206719 2553 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ece35fe8-abce-464f-a888-9798a5725a40-registry-certificates\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:42.207003 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.206728 2553 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ece35fe8-abce-464f-a888-9798a5725a40-installation-pull-secrets\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:42.207003 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.206737 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdw9k\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-kube-api-access-pdw9k\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:42.207003 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.206746 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-registry-tls\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:42.207003 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.206754 2553 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ece35fe8-abce-464f-a888-9798a5725a40-bound-sa-token\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:42.207003 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:42.206761 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ece35fe8-abce-464f-a888-9798a5725a40-trusted-ca\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:43.019090 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:43.019053 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" Sep 09 21:19:43.019572 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:43.019124 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-94b4b97c6-jlwgh" event={"ID":"ece35fe8-abce-464f-a888-9798a5725a40","Type":"ContainerDied","Data":"c83a36e4c58b71af63fced79a43525067e9654e1697178ae143385fd9d73a87b"} Sep 09 21:19:43.019572 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:43.019171 2553 scope.go:117] "RemoveContainer" containerID="0e85901e0972abc0f701aaf6ed77c73ece10ed44c7ca9bce0e71fee2e5493422" Sep 09 21:19:43.036939 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:43.036913 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-94b4b97c6-jlwgh"] Sep 09 21:19:43.040520 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:43.040496 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-94b4b97c6-jlwgh"] Sep 09 21:19:44.028117 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.028042 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86b4b89844-vklwg" podUID="21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" containerName="console" containerID="cri-o://80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6" gracePeriod=15 Sep 09 21:19:44.274384 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.274361 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:44.325858 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.325777 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-config\") pod \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " Sep 09 21:19:44.325858 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.325822 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-serving-cert\") pod \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " Sep 09 21:19:44.326040 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.325861 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-trusted-ca-bundle\") pod \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " Sep 09 21:19:44.326040 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.325891 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-service-ca\") pod \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " Sep 09 21:19:44.326040 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.325915 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-oauth-serving-cert\") pod \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " Sep 09 21:19:44.326040 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.325942 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzkzd\" (UniqueName: \"kubernetes.io/projected/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-kube-api-access-bzkzd\") pod \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " Sep 09 21:19:44.326265 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.326125 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-oauth-config\") pod \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\" (UID: \"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a\") " Sep 09 21:19:44.326337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.326307 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-config" (OuterVolumeSpecName: "console-config") pod "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" (UID: "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:19:44.326417 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.326391 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-service-ca" (OuterVolumeSpecName: "service-ca") pod "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" (UID: "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:19:44.326474 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.326408 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" (UID: "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:19:44.326474 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.326400 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" (UID: "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:19:44.326578 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.326511 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-config\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:44.326578 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.326533 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-trusted-ca-bundle\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:44.326578 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.326550 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-service-ca\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:44.326578 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.326564 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-oauth-serving-cert\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:44.328266 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.328240 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-kube-api-access-bzkzd" (OuterVolumeSpecName: "kube-api-access-bzkzd") pod "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" (UID: "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a"). InnerVolumeSpecName "kube-api-access-bzkzd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:19:44.328361 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.328276 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" (UID: "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:19:44.328361 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.328294 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" (UID: "21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:19:44.389897 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.389864 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ece35fe8-abce-464f-a888-9798a5725a40" path="/var/lib/kubelet/pods/ece35fe8-abce-464f-a888-9798a5725a40/volumes" Sep 09 21:19:44.427682 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.427651 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzkzd\" (UniqueName: \"kubernetes.io/projected/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-kube-api-access-bzkzd\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:44.427682 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.427682 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-oauth-config\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:44.427873 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:44.427695 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a-console-serving-cert\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:45.030221 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:45.030184 2553 generic.go:356] "Generic (PLEG): container finished" podID="21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" containerID="80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6" exitCode=0 Sep 09 21:19:45.030627 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:45.030228 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b4b89844-vklwg" Sep 09 21:19:45.030627 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:45.030261 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b4b89844-vklwg" event={"ID":"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a","Type":"ContainerDied","Data":"80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6"} Sep 09 21:19:45.030627 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:45.030304 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b4b89844-vklwg" event={"ID":"21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a","Type":"ContainerDied","Data":"4681092c358c3c7badf7bb44aa09d27331eb6b581dd3ef5635680f1366255396"} Sep 09 21:19:45.030627 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:45.030326 2553 scope.go:117] "RemoveContainer" containerID="80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6" Sep 09 21:19:45.038997 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:45.038976 2553 scope.go:117] "RemoveContainer" containerID="80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6" Sep 09 21:19:45.039401 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:19:45.039378 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6\": container with ID starting with 80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6 not found: ID does not exist" containerID="80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6" Sep 09 21:19:45.039461 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:45.039410 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6"} err="failed to get container status \"80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6\": rpc error: code = NotFound desc = could not find container \"80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6\": container with ID starting with 80b1013ad98286343a4fd9417ab519aac4e4e4f474592f571f481efc364c4cb6 not found: ID does not exist" Sep 09 21:19:45.047225 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:45.047200 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86b4b89844-vklwg"] Sep 09 21:19:45.051150 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:45.051126 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86b4b89844-vklwg"] Sep 09 21:19:46.389776 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:46.389740 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" path="/var/lib/kubelet/pods/21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a/volumes" Sep 09 21:19:50.356525 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:50.356496 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:19:50.357495 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:50.357474 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:19:54.978433 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:54.978384 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7ff4d48dfc-cnc6f" podUID="cc340dd7-5dce-4186-8e38-18e1e318dc7c" containerName="console" containerID="cri-o://636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9" gracePeriod=15 Sep 09 21:19:55.265004 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.264976 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:19:55.325305 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.325268 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-config\") pod \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " Sep 09 21:19:55.325486 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.325325 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-oauth-config\") pod \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " Sep 09 21:19:55.325486 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.325403 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-oauth-serving-cert\") pod \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " Sep 09 21:19:55.325486 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.325450 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ws77x\" (UniqueName: \"kubernetes.io/projected/cc340dd7-5dce-4186-8e38-18e1e318dc7c-kube-api-access-ws77x\") pod \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " Sep 09 21:19:55.325486 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.325471 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-service-ca\") pod \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " Sep 09 21:19:55.325687 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.325492 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-serving-cert\") pod \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " Sep 09 21:19:55.325687 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.325524 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-trusted-ca-bundle\") pod \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\" (UID: \"cc340dd7-5dce-4186-8e38-18e1e318dc7c\") " Sep 09 21:19:55.325841 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.325790 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-config" (OuterVolumeSpecName: "console-config") pod "cc340dd7-5dce-4186-8e38-18e1e318dc7c" (UID: "cc340dd7-5dce-4186-8e38-18e1e318dc7c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:19:55.326008 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.325839 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cc340dd7-5dce-4186-8e38-18e1e318dc7c" (UID: "cc340dd7-5dce-4186-8e38-18e1e318dc7c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:19:55.326008 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.325926 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-service-ca" (OuterVolumeSpecName: "service-ca") pod "cc340dd7-5dce-4186-8e38-18e1e318dc7c" (UID: "cc340dd7-5dce-4186-8e38-18e1e318dc7c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:19:55.326008 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.325935 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cc340dd7-5dce-4186-8e38-18e1e318dc7c" (UID: "cc340dd7-5dce-4186-8e38-18e1e318dc7c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:19:55.327605 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.327582 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc340dd7-5dce-4186-8e38-18e1e318dc7c-kube-api-access-ws77x" (OuterVolumeSpecName: "kube-api-access-ws77x") pod "cc340dd7-5dce-4186-8e38-18e1e318dc7c" (UID: "cc340dd7-5dce-4186-8e38-18e1e318dc7c"). InnerVolumeSpecName "kube-api-access-ws77x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:19:55.327699 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.327613 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cc340dd7-5dce-4186-8e38-18e1e318dc7c" (UID: "cc340dd7-5dce-4186-8e38-18e1e318dc7c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:19:55.327699 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.327689 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cc340dd7-5dce-4186-8e38-18e1e318dc7c" (UID: "cc340dd7-5dce-4186-8e38-18e1e318dc7c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:19:55.427113 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.427074 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws77x\" (UniqueName: \"kubernetes.io/projected/cc340dd7-5dce-4186-8e38-18e1e318dc7c-kube-api-access-ws77x\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:55.427292 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.427136 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-service-ca\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:55.427292 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.427148 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-serving-cert\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:55.427292 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.427158 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-trusted-ca-bundle\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:55.427292 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.427166 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-config\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:55.427292 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.427175 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cc340dd7-5dce-4186-8e38-18e1e318dc7c-console-oauth-config\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:55.427292 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:55.427185 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cc340dd7-5dce-4186-8e38-18e1e318dc7c-oauth-serving-cert\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:19:56.086300 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:56.086259 2553 generic.go:356] "Generic (PLEG): container finished" podID="cc340dd7-5dce-4186-8e38-18e1e318dc7c" containerID="636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9" exitCode=0 Sep 09 21:19:56.086776 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:56.086314 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ff4d48dfc-cnc6f" Sep 09 21:19:56.086776 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:56.086348 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff4d48dfc-cnc6f" event={"ID":"cc340dd7-5dce-4186-8e38-18e1e318dc7c","Type":"ContainerDied","Data":"636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9"} Sep 09 21:19:56.086776 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:56.086402 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ff4d48dfc-cnc6f" event={"ID":"cc340dd7-5dce-4186-8e38-18e1e318dc7c","Type":"ContainerDied","Data":"f388ac29e1caeb1adbba8d2596949fdc43ce5df5b4f22bf027c710fd2d9490e6"} Sep 09 21:19:56.086776 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:56.086424 2553 scope.go:117] "RemoveContainer" containerID="636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9" Sep 09 21:19:56.095703 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:56.095575 2553 scope.go:117] "RemoveContainer" containerID="636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9" Sep 09 21:19:56.095978 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:19:56.095954 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9\": container with ID starting with 636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9 not found: ID does not exist" containerID="636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9" Sep 09 21:19:56.096064 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:56.095989 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9"} err="failed to get container status \"636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9\": rpc error: code = NotFound desc = could not find container \"636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9\": container with ID starting with 636d3e331beaa37d5f2b83834ef02e8407937ac0aae0b448320d973313f5a5d9 not found: ID does not exist" Sep 09 21:19:56.110214 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:56.110183 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ff4d48dfc-cnc6f"] Sep 09 21:19:56.113754 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:56.113727 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7ff4d48dfc-cnc6f"] Sep 09 21:19:56.390350 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:19:56.390259 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc340dd7-5dce-4186-8e38-18e1e318dc7c" path="/var/lib/kubelet/pods/cc340dd7-5dce-4186-8e38-18e1e318dc7c/volumes" Sep 09 21:20:10.159787 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:20:10.159755 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:20:10.175556 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:20:10.175529 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:20:11.168755 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:20:11.168723 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Sep 09 21:24:50.389598 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:24:50.389530 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:24:50.389598 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:24:50.389537 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:25:32.410797 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.410757 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt"] Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411320 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411334 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411343 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411348 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411357 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411362 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411379 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc340dd7-5dce-4186-8e38-18e1e318dc7c" containerName="console" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411385 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc340dd7-5dce-4186-8e38-18e1e318dc7c" containerName="console" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411396 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" containerName="console" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411401 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" containerName="console" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411410 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411415 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411424 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411429 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411478 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="21aa8aa5-7beb-4db4-9f0e-b904ddeaa71a" containerName="console" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411486 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411493 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411499 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc340dd7-5dce-4186-8e38-18e1e318dc7c" containerName="console" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411505 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411510 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411515 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411566 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411571 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.413401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.411639 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:32.414542 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.414521 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.417513 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.417480 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7vrpf\"" Sep 09 21:25:32.417626 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.417548 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Sep 09 21:25:32.417681 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.417645 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Sep 09 21:25:32.418744 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.418709 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Sep 09 21:25:32.419041 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.419028 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Sep 09 21:25:32.430925 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.430897 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt"] Sep 09 21:25:32.507487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.507451 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a66d6957-d93d-4562-bb3a-e2504df76a07-manifests\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.507671 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.507557 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a66d6957-d93d-4562-bb3a-e2504df76a07-tmp\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.507671 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.507594 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a66d6957-d93d-4562-bb3a-e2504df76a07-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.507671 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.507622 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9fhv\" (UniqueName: \"kubernetes.io/projected/a66d6957-d93d-4562-bb3a-e2504df76a07-kube-api-access-b9fhv\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.507671 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.507642 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a66d6957-d93d-4562-bb3a-e2504df76a07-webhook-cert\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.608768 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.608729 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a66d6957-d93d-4562-bb3a-e2504df76a07-tmp\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.608768 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.608769 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a66d6957-d93d-4562-bb3a-e2504df76a07-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.609037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.608794 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9fhv\" (UniqueName: \"kubernetes.io/projected/a66d6957-d93d-4562-bb3a-e2504df76a07-kube-api-access-b9fhv\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.609037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.608815 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a66d6957-d93d-4562-bb3a-e2504df76a07-webhook-cert\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.609037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.608881 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a66d6957-d93d-4562-bb3a-e2504df76a07-manifests\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.609287 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.609266 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a66d6957-d93d-4562-bb3a-e2504df76a07-manifests\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.611132 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.611088 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a66d6957-d93d-4562-bb3a-e2504df76a07-tmp\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.611274 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.611255 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a66d6957-d93d-4562-bb3a-e2504df76a07-apiservice-cert\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.611336 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.611255 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a66d6957-d93d-4562-bb3a-e2504df76a07-webhook-cert\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.620450 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.620416 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9fhv\" (UniqueName: \"kubernetes.io/projected/a66d6957-d93d-4562-bb3a-e2504df76a07-kube-api-access-b9fhv\") pod \"opendatahub-operator-controller-manager-66dbc47bfc-nqxgt\" (UID: \"a66d6957-d93d-4562-bb3a-e2504df76a07\") " pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:32.726381 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:32.726340 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:33.008377 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:33.008347 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt"] Sep 09 21:25:33.010290 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:25:33.010261 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda66d6957_d93d_4562_bb3a_e2504df76a07.slice/crio-c0503c661a8d6a7856fc6c51362224e5fdd8516f6ffa24568a6231bee9afeb86 WatchSource:0}: Error finding container c0503c661a8d6a7856fc6c51362224e5fdd8516f6ffa24568a6231bee9afeb86: Status 404 returned error can't find the container with id c0503c661a8d6a7856fc6c51362224e5fdd8516f6ffa24568a6231bee9afeb86 Sep 09 21:25:33.012080 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:33.012060 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:25:33.638660 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:33.638619 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" event={"ID":"a66d6957-d93d-4562-bb3a-e2504df76a07","Type":"ContainerStarted","Data":"c0503c661a8d6a7856fc6c51362224e5fdd8516f6ffa24568a6231bee9afeb86"} Sep 09 21:25:36.654457 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:36.654420 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" event={"ID":"a66d6957-d93d-4562-bb3a-e2504df76a07","Type":"ContainerStarted","Data":"cc86d4a3b44994e10a02c04cda3e09edf2793a0cf89fb628735d97388c8bcecf"} Sep 09 21:25:37.660929 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:37.660837 2553 generic.go:356] "Generic (PLEG): container finished" podID="a66d6957-d93d-4562-bb3a-e2504df76a07" containerID="cc86d4a3b44994e10a02c04cda3e09edf2793a0cf89fb628735d97388c8bcecf" exitCode=0 Sep 09 21:25:37.660929 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:37.660878 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" event={"ID":"a66d6957-d93d-4562-bb3a-e2504df76a07","Type":"ContainerDied","Data":"cc86d4a3b44994e10a02c04cda3e09edf2793a0cf89fb628735d97388c8bcecf"} Sep 09 21:25:38.667491 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:38.667449 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" event={"ID":"a66d6957-d93d-4562-bb3a-e2504df76a07","Type":"ContainerStarted","Data":"7ffddec00a4bacd5e7c318ac704dc5f5bf5146144889533bf85feef3bf4208f5"} Sep 09 21:25:38.667891 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:38.667618 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:38.690843 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:38.690794 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" podStartSLOduration=2.156061227 podStartE2EDuration="6.690779884s" totalImagesPullingTime="3.494969833s" totalInitContainerRuntime="1.039748824s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:25:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:25:38.6890064 +0000 UTC m=+948.795750254" watchObservedRunningTime="2026-09-09 21:25:38.690779884 +0000 UTC m=+948.797523711" Sep 09 21:25:49.675179 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:49.675150 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-66dbc47bfc-nqxgt" Sep 09 21:25:51.790232 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.790193 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-696d6796dd-thz62"] Sep 09 21:25:51.790613 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.790560 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:51.790613 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.790571 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:51.790689 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.790618 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece35fe8-abce-464f-a888-9798a5725a40" containerName="registry" Sep 09 21:25:51.798044 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.798017 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" Sep 09 21:25:51.801292 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.801269 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Sep 09 21:25:51.801292 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.801284 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hwzww\"" Sep 09 21:25:51.801574 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.801550 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Sep 09 21:25:51.805241 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.805217 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-696d6796dd-thz62"] Sep 09 21:25:51.882576 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.882538 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51a8f793-9fae-4a02-bfc3-c5e42c7aab94-tls-certs\") pod \"kube-auth-proxy-696d6796dd-thz62\" (UID: \"51a8f793-9fae-4a02-bfc3-c5e42c7aab94\") " pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" Sep 09 21:25:51.882749 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.882597 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51a8f793-9fae-4a02-bfc3-c5e42c7aab94-tmp\") pod \"kube-auth-proxy-696d6796dd-thz62\" (UID: \"51a8f793-9fae-4a02-bfc3-c5e42c7aab94\") " pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" Sep 09 21:25:51.882749 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.882650 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njdp9\" (UniqueName: \"kubernetes.io/projected/51a8f793-9fae-4a02-bfc3-c5e42c7aab94-kube-api-access-njdp9\") pod \"kube-auth-proxy-696d6796dd-thz62\" (UID: \"51a8f793-9fae-4a02-bfc3-c5e42c7aab94\") " pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" Sep 09 21:25:51.983824 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.983789 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51a8f793-9fae-4a02-bfc3-c5e42c7aab94-tls-certs\") pod \"kube-auth-proxy-696d6796dd-thz62\" (UID: \"51a8f793-9fae-4a02-bfc3-c5e42c7aab94\") " pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" Sep 09 21:25:51.983996 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.983847 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51a8f793-9fae-4a02-bfc3-c5e42c7aab94-tmp\") pod \"kube-auth-proxy-696d6796dd-thz62\" (UID: \"51a8f793-9fae-4a02-bfc3-c5e42c7aab94\") " pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" Sep 09 21:25:51.983996 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.983875 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njdp9\" (UniqueName: \"kubernetes.io/projected/51a8f793-9fae-4a02-bfc3-c5e42c7aab94-kube-api-access-njdp9\") pod \"kube-auth-proxy-696d6796dd-thz62\" (UID: \"51a8f793-9fae-4a02-bfc3-c5e42c7aab94\") " pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" Sep 09 21:25:51.983996 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:25:51.983946 2553 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Sep 09 21:25:51.984128 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:25:51.984014 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51a8f793-9fae-4a02-bfc3-c5e42c7aab94-tls-certs podName:51a8f793-9fae-4a02-bfc3-c5e42c7aab94 nodeName:}" failed. No retries permitted until 2026-09-09 21:25:52.48399429 +0000 UTC m=+962.590738113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/51a8f793-9fae-4a02-bfc3-c5e42c7aab94-tls-certs") pod "kube-auth-proxy-696d6796dd-thz62" (UID: "51a8f793-9fae-4a02-bfc3-c5e42c7aab94") : secret "kube-auth-proxy-tls" not found Sep 09 21:25:51.986018 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:51.986000 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51a8f793-9fae-4a02-bfc3-c5e42c7aab94-tmp\") pod \"kube-auth-proxy-696d6796dd-thz62\" (UID: \"51a8f793-9fae-4a02-bfc3-c5e42c7aab94\") " pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" Sep 09 21:25:52.001351 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:52.001329 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njdp9\" (UniqueName: \"kubernetes.io/projected/51a8f793-9fae-4a02-bfc3-c5e42c7aab94-kube-api-access-njdp9\") pod \"kube-auth-proxy-696d6796dd-thz62\" (UID: \"51a8f793-9fae-4a02-bfc3-c5e42c7aab94\") " pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" Sep 09 21:25:52.488200 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:52.488162 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51a8f793-9fae-4a02-bfc3-c5e42c7aab94-tls-certs\") pod \"kube-auth-proxy-696d6796dd-thz62\" (UID: \"51a8f793-9fae-4a02-bfc3-c5e42c7aab94\") " pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" Sep 09 21:25:52.490516 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:52.490494 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51a8f793-9fae-4a02-bfc3-c5e42c7aab94-tls-certs\") pod \"kube-auth-proxy-696d6796dd-thz62\" (UID: \"51a8f793-9fae-4a02-bfc3-c5e42c7aab94\") " pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" Sep 09 21:25:52.708581 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:52.708538 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" Sep 09 21:25:52.996189 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:52.996161 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-696d6796dd-thz62"] Sep 09 21:25:52.998154 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:25:52.998121 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a8f793_9fae_4a02_bfc3_c5e42c7aab94.slice/crio-244550172e921d54b446bcf20b22f00cb86203935dbce8de71a0554d6b96c789 WatchSource:0}: Error finding container 244550172e921d54b446bcf20b22f00cb86203935dbce8de71a0554d6b96c789: Status 404 returned error can't find the container with id 244550172e921d54b446bcf20b22f00cb86203935dbce8de71a0554d6b96c789 Sep 09 21:25:53.205144 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.205086 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r"] Sep 09 21:25:53.209502 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.209482 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.212201 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.212173 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Sep 09 21:25:53.212282 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.212215 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 09 21:25:53.212334 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.212176 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zwxc2\"" Sep 09 21:25:53.212334 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.212172 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Sep 09 21:25:53.212595 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.212581 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Sep 09 21:25:53.231460 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.231432 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r"] Sep 09 21:25:53.295834 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.295798 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.295834 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.295837 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.296053 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.295857 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhqx8\" (UniqueName: \"kubernetes.io/projected/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-kube-api-access-qhqx8\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.296053 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.295878 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.296053 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.295916 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.296053 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.295953 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.296053 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.295982 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.396839 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.396804 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.396839 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.396845 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.397082 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.396904 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.397082 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.396923 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.397082 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.396940 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhqx8\" (UniqueName: \"kubernetes.io/projected/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-kube-api-access-qhqx8\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.397082 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.396959 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.397082 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.397030 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.397763 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.397715 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.399323 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.399303 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.399449 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.399302 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.399449 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.399403 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.399556 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.399512 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.405764 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.405738 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.405847 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.405804 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhqx8\" (UniqueName: \"kubernetes.io/projected/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-kube-api-access-qhqx8\") pod \"istiod-openshift-gateway-7b7995d5cd-trr5r\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.519190 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.519085 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:53.750118 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.750041 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" event={"ID":"51a8f793-9fae-4a02-bfc3-c5e42c7aab94","Type":"ContainerStarted","Data":"244550172e921d54b446bcf20b22f00cb86203935dbce8de71a0554d6b96c789"} Sep 09 21:25:53.812944 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:53.812913 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r"] Sep 09 21:25:53.815081 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:25:53.815049 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3b047f4_dc7d_4706_9b17_d38ebbaccc65.slice/crio-b8b434aaccc482e8f2301c068525c829af462f81716eb3d8d51c154f9e8aa868 WatchSource:0}: Error finding container b8b434aaccc482e8f2301c068525c829af462f81716eb3d8d51c154f9e8aa868: Status 404 returned error can't find the container with id b8b434aaccc482e8f2301c068525c829af462f81716eb3d8d51c154f9e8aa868 Sep 09 21:25:54.757178 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:54.757127 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" event={"ID":"b3b047f4-dc7d-4706-9b17-d38ebbaccc65","Type":"ContainerStarted","Data":"b8b434aaccc482e8f2301c068525c829af462f81716eb3d8d51c154f9e8aa868"} Sep 09 21:25:56.974676 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:56.974634 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 21:25:56.975064 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:56.974726 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 21:25:57.779574 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:57.779537 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" event={"ID":"b3b047f4-dc7d-4706-9b17-d38ebbaccc65","Type":"ContainerStarted","Data":"084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c"} Sep 09 21:25:57.779797 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:57.779698 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:25:57.781147 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:57.781124 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" event={"ID":"51a8f793-9fae-4a02-bfc3-c5e42c7aab94","Type":"ContainerStarted","Data":"db6d530a57748b872d5d76d2a9515e32d1685b9cf2115fcec1a506ab27838567"} Sep 09 21:25:57.802339 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:57.802291 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" podStartSLOduration=1.645663811 podStartE2EDuration="4.802275866s" totalImagesPullingTime="3.156612055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:25:57.799745241 +0000 UTC m=+967.906489081" watchObservedRunningTime="2026-09-09 21:25:57.802275866 +0000 UTC m=+967.909019691" Sep 09 21:25:57.815359 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:57.815314 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-696d6796dd-thz62" podStartSLOduration=2.888052706 podStartE2EDuration="6.815299336s" totalImagesPullingTime="3.92724663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:25:57.815038432 +0000 UTC m=+967.921782280" watchObservedRunningTime="2026-09-09 21:25:57.815299336 +0000 UTC m=+967.922043162" Sep 09 21:25:58.788437 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:58.788404 2553 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-trr5r container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Sep 09 21:25:58.788883 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:25:58.788461 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" podUID="b3b047f4-dc7d-4706-9b17-d38ebbaccc65" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Sep 09 21:26:01.787494 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:01.787458 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:26:13.697724 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.697644 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lqck6"] Sep 09 21:26:13.702319 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.702298 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" Sep 09 21:26:13.707848 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.707823 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Sep 09 21:26:13.711622 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.711599 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-lmr56\"" Sep 09 21:26:13.726035 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.726006 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lqck6"] Sep 09 21:26:13.728830 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.728803 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn"] Sep 09 21:26:13.732166 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.732151 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" Sep 09 21:26:13.737248 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.737217 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pbjvj\"" Sep 09 21:26:13.737898 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.737879 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Sep 09 21:26:13.770215 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.770180 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn"] Sep 09 21:26:13.781057 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.781025 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7z79\" (UniqueName: \"kubernetes.io/projected/43cda0ce-a5ed-411b-b506-76e212d3bafc-kube-api-access-m7z79\") pod \"kserve-controller-manager-d6f4b65b8-lqck6\" (UID: \"43cda0ce-a5ed-411b-b506-76e212d3bafc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" Sep 09 21:26:13.781258 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.781205 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grv4x\" (UniqueName: \"kubernetes.io/projected/4537c0f1-9d46-4c75-bcb2-1996f5497127-kube-api-access-grv4x\") pod \"llmisvc-controller-manager-777c6bd7d5-vj7dn\" (UID: \"4537c0f1-9d46-4c75-bcb2-1996f5497127\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" Sep 09 21:26:13.781258 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.781248 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43cda0ce-a5ed-411b-b506-76e212d3bafc-cert\") pod \"kserve-controller-manager-d6f4b65b8-lqck6\" (UID: \"43cda0ce-a5ed-411b-b506-76e212d3bafc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" Sep 09 21:26:13.781345 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.781272 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4537c0f1-9d46-4c75-bcb2-1996f5497127-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vj7dn\" (UID: \"4537c0f1-9d46-4c75-bcb2-1996f5497127\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" Sep 09 21:26:13.882012 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.881975 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grv4x\" (UniqueName: \"kubernetes.io/projected/4537c0f1-9d46-4c75-bcb2-1996f5497127-kube-api-access-grv4x\") pod \"llmisvc-controller-manager-777c6bd7d5-vj7dn\" (UID: \"4537c0f1-9d46-4c75-bcb2-1996f5497127\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" Sep 09 21:26:13.882012 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.882016 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43cda0ce-a5ed-411b-b506-76e212d3bafc-cert\") pod \"kserve-controller-manager-d6f4b65b8-lqck6\" (UID: \"43cda0ce-a5ed-411b-b506-76e212d3bafc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" Sep 09 21:26:13.882306 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.882038 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4537c0f1-9d46-4c75-bcb2-1996f5497127-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vj7dn\" (UID: \"4537c0f1-9d46-4c75-bcb2-1996f5497127\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" Sep 09 21:26:13.882306 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.882169 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7z79\" (UniqueName: \"kubernetes.io/projected/43cda0ce-a5ed-411b-b506-76e212d3bafc-kube-api-access-m7z79\") pod \"kserve-controller-manager-d6f4b65b8-lqck6\" (UID: \"43cda0ce-a5ed-411b-b506-76e212d3bafc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" Sep 09 21:26:13.882306 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:26:13.882221 2553 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Sep 09 21:26:13.882306 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:26:13.882299 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4537c0f1-9d46-4c75-bcb2-1996f5497127-cert podName:4537c0f1-9d46-4c75-bcb2-1996f5497127 nodeName:}" failed. No retries permitted until 2026-09-09 21:26:14.382275687 +0000 UTC m=+984.489019506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/4537c0f1-9d46-4c75-bcb2-1996f5497127-cert") pod "llmisvc-controller-manager-777c6bd7d5-vj7dn" (UID: "4537c0f1-9d46-4c75-bcb2-1996f5497127") : secret "llmisvc-webhook-server-cert" not found Sep 09 21:26:13.884660 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.884630 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43cda0ce-a5ed-411b-b506-76e212d3bafc-cert\") pod \"kserve-controller-manager-d6f4b65b8-lqck6\" (UID: \"43cda0ce-a5ed-411b-b506-76e212d3bafc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" Sep 09 21:26:13.903640 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.903605 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grv4x\" (UniqueName: \"kubernetes.io/projected/4537c0f1-9d46-4c75-bcb2-1996f5497127-kube-api-access-grv4x\") pod \"llmisvc-controller-manager-777c6bd7d5-vj7dn\" (UID: \"4537c0f1-9d46-4c75-bcb2-1996f5497127\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" Sep 09 21:26:13.913620 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:13.913584 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7z79\" (UniqueName: \"kubernetes.io/projected/43cda0ce-a5ed-411b-b506-76e212d3bafc-kube-api-access-m7z79\") pod \"kserve-controller-manager-d6f4b65b8-lqck6\" (UID: \"43cda0ce-a5ed-411b-b506-76e212d3bafc\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" Sep 09 21:26:14.013439 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.013348 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" Sep 09 21:26:14.305264 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:26:14.305230 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43cda0ce_a5ed_411b_b506_76e212d3bafc.slice/crio-f8e63375ee0c4a54d018f31f7acea57a405cb014a0214b82f7e33217c0d8eb06 WatchSource:0}: Error finding container f8e63375ee0c4a54d018f31f7acea57a405cb014a0214b82f7e33217c0d8eb06: Status 404 returned error can't find the container with id f8e63375ee0c4a54d018f31f7acea57a405cb014a0214b82f7e33217c0d8eb06 Sep 09 21:26:14.305561 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.305540 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lqck6"] Sep 09 21:26:14.387250 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.387217 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4537c0f1-9d46-4c75-bcb2-1996f5497127-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vj7dn\" (UID: \"4537c0f1-9d46-4c75-bcb2-1996f5497127\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" Sep 09 21:26:14.389976 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.389948 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4537c0f1-9d46-4c75-bcb2-1996f5497127-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vj7dn\" (UID: \"4537c0f1-9d46-4c75-bcb2-1996f5497127\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" Sep 09 21:26:14.564633 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.564145 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r"] Sep 09 21:26:14.564903 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.564416 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" podUID="b3b047f4-dc7d-4706-9b17-d38ebbaccc65" containerName="discovery" containerID="cri-o://084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c" gracePeriod=30 Sep 09 21:26:14.642651 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.642613 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" Sep 09 21:26:14.822553 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.822492 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:26:14.874703 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.874672 2553 generic.go:356] "Generic (PLEG): container finished" podID="b3b047f4-dc7d-4706-9b17-d38ebbaccc65" containerID="084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c" exitCode=0 Sep 09 21:26:14.874951 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.874754 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" Sep 09 21:26:14.874951 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.874804 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" event={"ID":"b3b047f4-dc7d-4706-9b17-d38ebbaccc65","Type":"ContainerDied","Data":"084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c"} Sep 09 21:26:14.874951 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.874894 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r" event={"ID":"b3b047f4-dc7d-4706-9b17-d38ebbaccc65","Type":"ContainerDied","Data":"b8b434aaccc482e8f2301c068525c829af462f81716eb3d8d51c154f9e8aa868"} Sep 09 21:26:14.874951 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.874920 2553 scope.go:117] "RemoveContainer" containerID="084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c" Sep 09 21:26:14.876434 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.876414 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" event={"ID":"43cda0ce-a5ed-411b-b506-76e212d3bafc","Type":"ContainerStarted","Data":"f8e63375ee0c4a54d018f31f7acea57a405cb014a0214b82f7e33217c0d8eb06"} Sep 09 21:26:14.883915 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.883885 2553 scope.go:117] "RemoveContainer" containerID="084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c" Sep 09 21:26:14.884359 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:26:14.884339 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c\": container with ID starting with 084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c not found: ID does not exist" containerID="084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c" Sep 09 21:26:14.884426 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.884366 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c"} err="failed to get container status \"084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c\": rpc error: code = NotFound desc = could not find container \"084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c\": container with ID starting with 084aa5c3037981042860d0e90eff6c7655a340e5d693c1877e79390bc593da6c not found: ID does not exist" Sep 09 21:26:14.892728 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.892713 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-cacerts\") pod \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " Sep 09 21:26:14.892788 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.892751 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-csr-dns-cert\") pod \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " Sep 09 21:26:14.892832 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.892819 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhqx8\" (UniqueName: \"kubernetes.io/projected/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-kube-api-access-qhqx8\") pod \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " Sep 09 21:26:14.892885 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.892865 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-token\") pod \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " Sep 09 21:26:14.892938 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.892891 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-kubeconfig\") pod \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " Sep 09 21:26:14.892938 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.892925 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-csr-ca-configmap\") pod \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " Sep 09 21:26:14.893042 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.892963 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-local-certs\") pod \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\" (UID: \"b3b047f4-dc7d-4706-9b17-d38ebbaccc65\") " Sep 09 21:26:14.893654 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.893482 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b3b047f4-dc7d-4706-9b17-d38ebbaccc65" (UID: "b3b047f4-dc7d-4706-9b17-d38ebbaccc65"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:26:14.895799 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.895763 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-kube-api-access-qhqx8" (OuterVolumeSpecName: "kube-api-access-qhqx8") pod "b3b047f4-dc7d-4706-9b17-d38ebbaccc65" (UID: "b3b047f4-dc7d-4706-9b17-d38ebbaccc65"). InnerVolumeSpecName "kube-api-access-qhqx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:26:14.895894 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.895792 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-cacerts" (OuterVolumeSpecName: "cacerts") pod "b3b047f4-dc7d-4706-9b17-d38ebbaccc65" (UID: "b3b047f4-dc7d-4706-9b17-d38ebbaccc65"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:26:14.896037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.896011 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b3b047f4-dc7d-4706-9b17-d38ebbaccc65" (UID: "b3b047f4-dc7d-4706-9b17-d38ebbaccc65"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:26:14.896218 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.896062 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b3b047f4-dc7d-4706-9b17-d38ebbaccc65" (UID: "b3b047f4-dc7d-4706-9b17-d38ebbaccc65"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:26:14.896463 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.896409 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-local-certs" (OuterVolumeSpecName: "local-certs") pod "b3b047f4-dc7d-4706-9b17-d38ebbaccc65" (UID: "b3b047f4-dc7d-4706-9b17-d38ebbaccc65"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:26:14.896463 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.896424 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-token" (OuterVolumeSpecName: "istio-token") pod "b3b047f4-dc7d-4706-9b17-d38ebbaccc65" (UID: "b3b047f4-dc7d-4706-9b17-d38ebbaccc65"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:26:14.932020 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.931994 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn"] Sep 09 21:26:14.932510 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:26:14.932478 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4537c0f1_9d46_4c75_bcb2_1996f5497127.slice/crio-0eb25f8c975e621bcb759a6e18d3e70bae2fbbeea9792608be19e1c11b3c2ab7 WatchSource:0}: Error finding container 0eb25f8c975e621bcb759a6e18d3e70bae2fbbeea9792608be19e1c11b3c2ab7: Status 404 returned error can't find the container with id 0eb25f8c975e621bcb759a6e18d3e70bae2fbbeea9792608be19e1c11b3c2ab7 Sep 09 21:26:14.994189 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.994156 2553 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-kubeconfig\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:26:14.994189 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.994187 2553 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-csr-ca-configmap\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:26:14.994189 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.994198 2553 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-local-certs\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:26:14.994420 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.994209 2553 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-cacerts\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:26:14.994420 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.994217 2553 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-csr-dns-cert\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:26:14.994420 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.994226 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhqx8\" (UniqueName: \"kubernetes.io/projected/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-kube-api-access-qhqx8\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:26:14.994420 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:14.994234 2553 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b3b047f4-dc7d-4706-9b17-d38ebbaccc65-istio-token\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:26:15.200974 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:15.200948 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r"] Sep 09 21:26:15.205345 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:15.205319 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-trr5r"] Sep 09 21:26:15.885614 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:15.885259 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" event={"ID":"4537c0f1-9d46-4c75-bcb2-1996f5497127","Type":"ContainerStarted","Data":"0eb25f8c975e621bcb759a6e18d3e70bae2fbbeea9792608be19e1c11b3c2ab7"} Sep 09 21:26:16.392231 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:16.392190 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3b047f4-dc7d-4706-9b17-d38ebbaccc65" path="/var/lib/kubelet/pods/b3b047f4-dc7d-4706-9b17-d38ebbaccc65/volumes" Sep 09 21:26:17.899915 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:17.899823 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" event={"ID":"4537c0f1-9d46-4c75-bcb2-1996f5497127","Type":"ContainerStarted","Data":"222793d5fea835dd210be297d6b9e6d50189ae2d61d5a12b0111b6b82c7033a5"} Sep 09 21:26:17.900364 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:17.900004 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" Sep 09 21:26:17.901434 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:17.901414 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" event={"ID":"43cda0ce-a5ed-411b-b506-76e212d3bafc","Type":"ContainerStarted","Data":"b68ac92acc2770b4733f6b4caad13eb1c06df444252d60661d00a37be123d902"} Sep 09 21:26:17.901557 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:17.901544 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" Sep 09 21:26:17.917482 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:17.917426 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" podStartSLOduration=2.279529153 podStartE2EDuration="4.917408772s" totalImagesPullingTime="2.637879619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:26:17.91725157 +0000 UTC m=+988.023995421" watchObservedRunningTime="2026-09-09 21:26:17.917408772 +0000 UTC m=+988.024152598" Sep 09 21:26:17.933121 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:17.933060 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" podStartSLOduration=2.315756619 podStartE2EDuration="4.933044757s" totalImagesPullingTime="2.617288138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:26:17.932267242 +0000 UTC m=+988.039011088" watchObservedRunningTime="2026-09-09 21:26:17.933044757 +0000 UTC m=+988.039788585" Sep 09 21:26:48.908463 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:48.908420 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vj7dn" Sep 09 21:26:48.911932 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:26:48.911903 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lqck6" Sep 09 21:27:21.435890 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.435848 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-svmkk"] Sep 09 21:27:21.436473 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.436418 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3b047f4-dc7d-4706-9b17-d38ebbaccc65" containerName="discovery" Sep 09 21:27:21.436473 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.436438 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3b047f4-dc7d-4706-9b17-d38ebbaccc65" containerName="discovery" Sep 09 21:27:21.436590 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.436561 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3b047f4-dc7d-4706-9b17-d38ebbaccc65" containerName="discovery" Sep 09 21:27:21.439589 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.439570 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-svmkk" Sep 09 21:27:21.444516 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.444496 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Sep 09 21:27:21.444654 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.444531 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Sep 09 21:27:21.444654 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.444569 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-t485n\"" Sep 09 21:27:21.451925 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.451901 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-svmkk"] Sep 09 21:27:21.596859 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.596819 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrq25\" (UniqueName: \"kubernetes.io/projected/6a030bb6-6577-47e1-a370-aee5ccada074-kube-api-access-wrq25\") pod \"authorino-operator-76b478bd78-svmkk\" (UID: \"6a030bb6-6577-47e1-a370-aee5ccada074\") " pod="kuadrant-system/authorino-operator-76b478bd78-svmkk" Sep 09 21:27:21.697747 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.697663 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrq25\" (UniqueName: \"kubernetes.io/projected/6a030bb6-6577-47e1-a370-aee5ccada074-kube-api-access-wrq25\") pod \"authorino-operator-76b478bd78-svmkk\" (UID: \"6a030bb6-6577-47e1-a370-aee5ccada074\") " pod="kuadrant-system/authorino-operator-76b478bd78-svmkk" Sep 09 21:27:21.713058 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.713033 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrq25\" (UniqueName: \"kubernetes.io/projected/6a030bb6-6577-47e1-a370-aee5ccada074-kube-api-access-wrq25\") pod \"authorino-operator-76b478bd78-svmkk\" (UID: \"6a030bb6-6577-47e1-a370-aee5ccada074\") " pod="kuadrant-system/authorino-operator-76b478bd78-svmkk" Sep 09 21:27:21.750253 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:21.750216 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-svmkk" Sep 09 21:27:22.030862 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:22.030837 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-svmkk"] Sep 09 21:27:22.033423 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:27:22.033397 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a030bb6_6577_47e1_a370_aee5ccada074.slice/crio-b96604357986034dbaa5da43422faffecefb9f3aa6230ed84b684e1674ac407c WatchSource:0}: Error finding container b96604357986034dbaa5da43422faffecefb9f3aa6230ed84b684e1674ac407c: Status 404 returned error can't find the container with id b96604357986034dbaa5da43422faffecefb9f3aa6230ed84b684e1674ac407c Sep 09 21:27:22.216961 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:22.216929 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-svmkk" event={"ID":"6a030bb6-6577-47e1-a370-aee5ccada074","Type":"ContainerStarted","Data":"b96604357986034dbaa5da43422faffecefb9f3aa6230ed84b684e1674ac407c"} Sep 09 21:27:23.759603 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:23.759573 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq"] Sep 09 21:27:23.765074 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:23.765042 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq" Sep 09 21:27:23.767776 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:23.767748 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Sep 09 21:27:23.768291 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:23.768262 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-r6tfn\"" Sep 09 21:27:23.781277 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:23.781251 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq"] Sep 09 21:27:23.918483 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:23.918440 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9rhv\" (UniqueName: \"kubernetes.io/projected/be3faeaa-0256-4c16-9c07-76ead90b33ae-kube-api-access-l9rhv\") pod \"dns-operator-controller-manager-7b84cd7c9c-bkqqq\" (UID: \"be3faeaa-0256-4c16-9c07-76ead90b33ae\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq" Sep 09 21:27:24.019138 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:24.019044 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9rhv\" (UniqueName: \"kubernetes.io/projected/be3faeaa-0256-4c16-9c07-76ead90b33ae-kube-api-access-l9rhv\") pod \"dns-operator-controller-manager-7b84cd7c9c-bkqqq\" (UID: \"be3faeaa-0256-4c16-9c07-76ead90b33ae\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq" Sep 09 21:27:24.031158 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:24.031129 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9rhv\" (UniqueName: \"kubernetes.io/projected/be3faeaa-0256-4c16-9c07-76ead90b33ae-kube-api-access-l9rhv\") pod \"dns-operator-controller-manager-7b84cd7c9c-bkqqq\" (UID: \"be3faeaa-0256-4c16-9c07-76ead90b33ae\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq" Sep 09 21:27:24.089114 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:24.089063 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq" Sep 09 21:27:24.935779 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:24.935755 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq"] Sep 09 21:27:24.937539 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:27:24.937517 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe3faeaa_0256_4c16_9c07_76ead90b33ae.slice/crio-17874bc18305f2ab3eae8c6aad892efdb513728bd84df6a586b811249379fd85 WatchSource:0}: Error finding container 17874bc18305f2ab3eae8c6aad892efdb513728bd84df6a586b811249379fd85: Status 404 returned error can't find the container with id 17874bc18305f2ab3eae8c6aad892efdb513728bd84df6a586b811249379fd85 Sep 09 21:27:25.236177 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:25.236067 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-svmkk" event={"ID":"6a030bb6-6577-47e1-a370-aee5ccada074","Type":"ContainerStarted","Data":"42b5c1a8abf05cc843685e9d4b18c74e23837c03dc7100968bac6f87ee316766"} Sep 09 21:27:25.236177 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:25.236155 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-svmkk" Sep 09 21:27:25.237428 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:25.237403 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq" event={"ID":"be3faeaa-0256-4c16-9c07-76ead90b33ae","Type":"ContainerStarted","Data":"17874bc18305f2ab3eae8c6aad892efdb513728bd84df6a586b811249379fd85"} Sep 09 21:27:25.259021 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:25.258965 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-svmkk" podStartSLOduration=1.5602426299999999 podStartE2EDuration="4.258947638s" totalImagesPullingTime="2.698705008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:27:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:27:25.257974638 +0000 UTC m=+1055.364718485" watchObservedRunningTime="2026-09-09 21:27:25.258947638 +0000 UTC m=+1055.365691463" Sep 09 21:27:26.110584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:26.110546 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l"] Sep 09 21:27:26.114183 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:26.114160 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:26.118796 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:26.118773 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-bvg9d\"" Sep 09 21:27:26.129468 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:26.129438 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l"] Sep 09 21:27:26.246463 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:26.246426 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg5s7\" (UniqueName: \"kubernetes.io/projected/fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6-kube-api-access-jg5s7\") pod \"limitador-operator-controller-manager-74495c69f8-llb8l\" (UID: \"fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:26.347117 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:26.347058 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg5s7\" (UniqueName: \"kubernetes.io/projected/fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6-kube-api-access-jg5s7\") pod \"limitador-operator-controller-manager-74495c69f8-llb8l\" (UID: \"fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:26.359396 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:26.359365 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg5s7\" (UniqueName: \"kubernetes.io/projected/fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6-kube-api-access-jg5s7\") pod \"limitador-operator-controller-manager-74495c69f8-llb8l\" (UID: \"fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:26.428551 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:26.428517 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:26.837156 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:26.837131 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l"] Sep 09 21:27:26.838984 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:27:26.838963 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4a7a0b_895b_4ed2_8a00_7a2a22c6beb6.slice/crio-03c3ee9a3c56db0e79c4e7d97fda0653ff3d67985fec16a40104ad0bf885996f WatchSource:0}: Error finding container 03c3ee9a3c56db0e79c4e7d97fda0653ff3d67985fec16a40104ad0bf885996f: Status 404 returned error can't find the container with id 03c3ee9a3c56db0e79c4e7d97fda0653ff3d67985fec16a40104ad0bf885996f Sep 09 21:27:27.250716 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:27.250682 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" event={"ID":"fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6","Type":"ContainerStarted","Data":"03c3ee9a3c56db0e79c4e7d97fda0653ff3d67985fec16a40104ad0bf885996f"} Sep 09 21:27:27.252211 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:27.252190 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq" event={"ID":"be3faeaa-0256-4c16-9c07-76ead90b33ae","Type":"ContainerStarted","Data":"e6e4953f9b4236a9956e565e2a0bac8f3292ac1dd6e537027f3e43ffcc740e03"} Sep 09 21:27:27.252314 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:27.252304 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq" Sep 09 21:27:27.274952 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:27.274901 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq" podStartSLOduration=2.594552904 podStartE2EDuration="4.274883895s" totalImagesPullingTime="1.680330991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:27:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:27:27.273761041 +0000 UTC m=+1057.380504904" watchObservedRunningTime="2026-09-09 21:27:27.274883895 +0000 UTC m=+1057.381627718" Sep 09 21:27:29.264694 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:29.264647 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" event={"ID":"fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6","Type":"ContainerStarted","Data":"a07a3b9525ef4e416c61b2048b145236f058b56083b3780ce15bf19509d700f3"} Sep 09 21:27:29.265121 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:29.264741 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:29.283166 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:29.283093 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" podStartSLOduration=1.479174557 podStartE2EDuration="3.283078603s" totalImagesPullingTime="1.803904046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:27:29.281944171 +0000 UTC m=+1059.388687997" watchObservedRunningTime="2026-09-09 21:27:29.283078603 +0000 UTC m=+1059.389822430" Sep 09 21:27:30.300578 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:30.300546 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs"] Sep 09 21:27:30.304427 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:30.304405 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:30.307161 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:30.307140 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rb55z\"" Sep 09 21:27:30.321658 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:30.321616 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs"] Sep 09 21:27:30.482636 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:30.482586 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e67de1d-ae77-4772-beef-f28ec64b1647-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" (UID: \"2e67de1d-ae77-4772-beef-f28ec64b1647\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:30.482821 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:30.482656 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psjf4\" (UniqueName: \"kubernetes.io/projected/2e67de1d-ae77-4772-beef-f28ec64b1647-kube-api-access-psjf4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" (UID: \"2e67de1d-ae77-4772-beef-f28ec64b1647\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:30.584212 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:30.584116 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e67de1d-ae77-4772-beef-f28ec64b1647-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" (UID: \"2e67de1d-ae77-4772-beef-f28ec64b1647\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:30.584212 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:30.584172 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psjf4\" (UniqueName: \"kubernetes.io/projected/2e67de1d-ae77-4772-beef-f28ec64b1647-kube-api-access-psjf4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" (UID: \"2e67de1d-ae77-4772-beef-f28ec64b1647\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:30.584653 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:30.584612 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e67de1d-ae77-4772-beef-f28ec64b1647-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" (UID: \"2e67de1d-ae77-4772-beef-f28ec64b1647\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:30.593975 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:30.593947 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psjf4\" (UniqueName: \"kubernetes.io/projected/2e67de1d-ae77-4772-beef-f28ec64b1647-kube-api-access-psjf4\") pod \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" (UID: \"2e67de1d-ae77-4772-beef-f28ec64b1647\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:30.614848 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:30.614817 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:30.893788 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:30.893764 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs"] Sep 09 21:27:30.896562 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:27:30.896527 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e67de1d_ae77_4772_beef_f28ec64b1647.slice/crio-9b81884a300bda987c222f971d45c19f004c580b7b5c8993b30b3641e3b6e0e6 WatchSource:0}: Error finding container 9b81884a300bda987c222f971d45c19f004c580b7b5c8993b30b3641e3b6e0e6: Status 404 returned error can't find the container with id 9b81884a300bda987c222f971d45c19f004c580b7b5c8993b30b3641e3b6e0e6 Sep 09 21:27:31.275714 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:31.275676 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" event={"ID":"2e67de1d-ae77-4772-beef-f28ec64b1647","Type":"ContainerStarted","Data":"9b81884a300bda987c222f971d45c19f004c580b7b5c8993b30b3641e3b6e0e6"} Sep 09 21:27:35.305612 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:35.305580 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" event={"ID":"2e67de1d-ae77-4772-beef-f28ec64b1647","Type":"ContainerStarted","Data":"153ee0ba8d95574d5e1fe07e6ffb23ce072934a8e1588e798e695bd8d56b9ff1"} Sep 09 21:27:35.306040 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:35.305687 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:35.335185 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:35.335127 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" podStartSLOduration=1.059662688 podStartE2EDuration="5.335092942s" totalImagesPullingTime="4.275430254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:27:35.333360722 +0000 UTC m=+1065.440104572" watchObservedRunningTime="2026-09-09 21:27:35.335092942 +0000 UTC m=+1065.441836767" Sep 09 21:27:36.246582 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:36.246554 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-svmkk" Sep 09 21:27:38.261243 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:38.261213 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-bkqqq" Sep 09 21:27:40.272507 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:40.272478 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:46.313557 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:46.313480 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:47.454509 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:47.454474 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt"] Sep 09 21:27:47.458067 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:47.458052 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:47.477727 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:47.477698 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt"] Sep 09 21:27:47.536752 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:47.536717 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d8hz\" (UniqueName: \"kubernetes.io/projected/a9ab342e-dd20-4aec-957c-516c4c9c98b7-kube-api-access-7d8hz\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-rfstt\" (UID: \"a9ab342e-dd20-4aec-957c-516c4c9c98b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:47.536950 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:47.536833 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9ab342e-dd20-4aec-957c-516c4c9c98b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-rfstt\" (UID: \"a9ab342e-dd20-4aec-957c-516c4c9c98b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:47.638311 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:47.638270 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d8hz\" (UniqueName: \"kubernetes.io/projected/a9ab342e-dd20-4aec-957c-516c4c9c98b7-kube-api-access-7d8hz\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-rfstt\" (UID: \"a9ab342e-dd20-4aec-957c-516c4c9c98b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:47.638477 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:47.638429 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9ab342e-dd20-4aec-957c-516c4c9c98b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-rfstt\" (UID: \"a9ab342e-dd20-4aec-957c-516c4c9c98b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:47.638779 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:47.638762 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9ab342e-dd20-4aec-957c-516c4c9c98b7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-rfstt\" (UID: \"a9ab342e-dd20-4aec-957c-516c4c9c98b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:47.648275 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:47.648252 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d8hz\" (UniqueName: \"kubernetes.io/projected/a9ab342e-dd20-4aec-957c-516c4c9c98b7-kube-api-access-7d8hz\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-rfstt\" (UID: \"a9ab342e-dd20-4aec-957c-516c4c9c98b7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:47.768134 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:47.768037 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:48.041652 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.041624 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt"] Sep 09 21:27:48.043296 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:27:48.043271 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ab342e_dd20_4aec_957c_516c4c9c98b7.slice/crio-f46cff30e4197fcfd1099445042aa6c230657c10b42ddd4a3f23e9a4adf13e9e WatchSource:0}: Error finding container f46cff30e4197fcfd1099445042aa6c230657c10b42ddd4a3f23e9a4adf13e9e: Status 404 returned error can't find the container with id f46cff30e4197fcfd1099445042aa6c230657c10b42ddd4a3f23e9a4adf13e9e Sep 09 21:27:48.377265 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.377158 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" event={"ID":"a9ab342e-dd20-4aec-957c-516c4c9c98b7","Type":"ContainerStarted","Data":"dd0492e222d5d6b957d8051b824fc31e23c55e6f4988c22bf0b87f875942679f"} Sep 09 21:27:48.377265 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.377223 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" event={"ID":"a9ab342e-dd20-4aec-957c-516c4c9c98b7","Type":"ContainerStarted","Data":"f46cff30e4197fcfd1099445042aa6c230657c10b42ddd4a3f23e9a4adf13e9e"} Sep 09 21:27:48.377265 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.377248 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:48.400215 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.400164 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" podStartSLOduration=1.4001464399999999 podStartE2EDuration="1.40014644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:27:48.398452783 +0000 UTC m=+1078.505196639" watchObservedRunningTime="2026-09-09 21:27:48.40014644 +0000 UTC m=+1078.506890266" Sep 09 21:27:48.569376 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.569335 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt"] Sep 09 21:27:48.574870 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.573868 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt"] Sep 09 21:27:48.587018 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.586982 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs"] Sep 09 21:27:48.587257 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.587231 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" containerName="manager" containerID="cri-o://153ee0ba8d95574d5e1fe07e6ffb23ce072934a8e1588e798e695bd8d56b9ff1" gracePeriod=2 Sep 09 21:27:48.595194 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.595160 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc"] Sep 09 21:27:48.595812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.595781 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9ab342e-dd20-4aec-957c-516c4c9c98b7" containerName="manager" Sep 09 21:27:48.595812 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.595808 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9ab342e-dd20-4aec-957c-516c4c9c98b7" containerName="manager" Sep 09 21:27:48.595969 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.595907 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9ab342e-dd20-4aec-957c-516c4c9c98b7" containerName="manager" Sep 09 21:27:48.600396 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.600293 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" Sep 09 21:27:48.602057 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.602032 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs"] Sep 09 21:27:48.608867 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:48.608835 2553 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"2e67de1d-ae77-4772-beef-f28ec64b1647\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-vdggs\": pods \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:48.617670 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.617644 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l"] Sep 09 21:27:48.617852 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.617832 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" containerName="manager" containerID="cri-o://a07a3b9525ef4e416c61b2048b145236f058b56083b3780ce15bf19509d700f3" gracePeriod=2 Sep 09 21:27:48.620082 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.620057 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc"] Sep 09 21:27:48.624069 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.623977 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9"] Sep 09 21:27:48.624578 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.624460 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" containerName="manager" Sep 09 21:27:48.624578 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.624477 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" containerName="manager" Sep 09 21:27:48.624578 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.624560 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" containerName="manager" Sep 09 21:27:48.627567 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.627514 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l"] Sep 09 21:27:48.627653 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.627610 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" Sep 09 21:27:48.643875 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.642277 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9"] Sep 09 21:27:48.648170 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:48.648070 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:48.648764 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.648538 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05ce3699-b6cc-4239-b4a6-406a037de1d5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-g6kfc\" (UID: \"05ce3699-b6cc-4239-b4a6-406a037de1d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" Sep 09 21:27:48.648764 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.648595 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4fq4\" (UniqueName: \"kubernetes.io/projected/05ce3699-b6cc-4239-b4a6-406a037de1d5-kube-api-access-w4fq4\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-g6kfc\" (UID: \"05ce3699-b6cc-4239-b4a6-406a037de1d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" Sep 09 21:27:48.649740 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.649690 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp"] Sep 09 21:27:48.650447 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.650359 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" containerName="manager" Sep 09 21:27:48.650447 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.650375 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" containerName="manager" Sep 09 21:27:48.650641 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.650466 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" containerName="manager" Sep 09 21:27:48.650998 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:48.650839 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:48.654066 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.654048 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp" Sep 09 21:27:48.664495 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.664471 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp"] Sep 09 21:27:48.676883 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:48.676840 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-llb8l\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:48.678957 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:48.678923 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:48.702000 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:48.701962 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-llb8l\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:48.749272 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.749237 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcqw4\" (UniqueName: \"kubernetes.io/projected/23aad71c-05cb-4a06-a24c-764c9ea1cb1e-kube-api-access-fcqw4\") pod \"limitador-operator-controller-manager-74495c69f8-wgfgp\" (UID: \"23aad71c-05cb-4a06-a24c-764c9ea1cb1e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp" Sep 09 21:27:48.749272 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.749276 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dca8eeb0-1289-4179-9f43-54479fd61bb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8nbx9\" (UID: \"dca8eeb0-1289-4179-9f43-54479fd61bb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" Sep 09 21:27:48.749531 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.749450 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05ce3699-b6cc-4239-b4a6-406a037de1d5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-g6kfc\" (UID: \"05ce3699-b6cc-4239-b4a6-406a037de1d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" Sep 09 21:27:48.749531 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.749508 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4fq4\" (UniqueName: \"kubernetes.io/projected/05ce3699-b6cc-4239-b4a6-406a037de1d5-kube-api-access-w4fq4\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-g6kfc\" (UID: \"05ce3699-b6cc-4239-b4a6-406a037de1d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" Sep 09 21:27:48.749649 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.749574 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5sf7\" (UniqueName: \"kubernetes.io/projected/dca8eeb0-1289-4179-9f43-54479fd61bb2-kube-api-access-d5sf7\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8nbx9\" (UID: \"dca8eeb0-1289-4179-9f43-54479fd61bb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" Sep 09 21:27:48.750039 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.749992 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05ce3699-b6cc-4239-b4a6-406a037de1d5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-g6kfc\" (UID: \"05ce3699-b6cc-4239-b4a6-406a037de1d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" Sep 09 21:27:48.760872 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.760839 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4fq4\" (UniqueName: \"kubernetes.io/projected/05ce3699-b6cc-4239-b4a6-406a037de1d5-kube-api-access-w4fq4\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-g6kfc\" (UID: \"05ce3699-b6cc-4239-b4a6-406a037de1d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" Sep 09 21:27:48.850532 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.850460 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5sf7\" (UniqueName: \"kubernetes.io/projected/dca8eeb0-1289-4179-9f43-54479fd61bb2-kube-api-access-d5sf7\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8nbx9\" (UID: \"dca8eeb0-1289-4179-9f43-54479fd61bb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" Sep 09 21:27:48.850587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.850538 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcqw4\" (UniqueName: \"kubernetes.io/projected/23aad71c-05cb-4a06-a24c-764c9ea1cb1e-kube-api-access-fcqw4\") pod \"limitador-operator-controller-manager-74495c69f8-wgfgp\" (UID: \"23aad71c-05cb-4a06-a24c-764c9ea1cb1e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp" Sep 09 21:27:48.850587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.850565 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dca8eeb0-1289-4179-9f43-54479fd61bb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8nbx9\" (UID: \"dca8eeb0-1289-4179-9f43-54479fd61bb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" Sep 09 21:27:48.850995 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.850966 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dca8eeb0-1289-4179-9f43-54479fd61bb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8nbx9\" (UID: \"dca8eeb0-1289-4179-9f43-54479fd61bb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" Sep 09 21:27:48.859600 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.859581 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcqw4\" (UniqueName: \"kubernetes.io/projected/23aad71c-05cb-4a06-a24c-764c9ea1cb1e-kube-api-access-fcqw4\") pod \"limitador-operator-controller-manager-74495c69f8-wgfgp\" (UID: \"23aad71c-05cb-4a06-a24c-764c9ea1cb1e\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp" Sep 09 21:27:48.859684 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.859659 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5sf7\" (UniqueName: \"kubernetes.io/projected/dca8eeb0-1289-4179-9f43-54479fd61bb2-kube-api-access-d5sf7\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8nbx9\" (UID: \"dca8eeb0-1289-4179-9f43-54479fd61bb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" Sep 09 21:27:48.869253 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.869226 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:48.871398 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.871379 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:48.871596 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:48.871574 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-llb8l\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:48.873737 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:48.873717 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:48.875802 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:48.875783 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:48.877967 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:48.877919 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-llb8l\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:48.951269 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.951231 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e67de1d-ae77-4772-beef-f28ec64b1647-extensions-socket-volume\") pod \"2e67de1d-ae77-4772-beef-f28ec64b1647\" (UID: \"2e67de1d-ae77-4772-beef-f28ec64b1647\") " Sep 09 21:27:48.951412 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.951352 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jg5s7\" (UniqueName: \"kubernetes.io/projected/fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6-kube-api-access-jg5s7\") pod \"fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6\" (UID: \"fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6\") " Sep 09 21:27:48.951412 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.951370 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psjf4\" (UniqueName: \"kubernetes.io/projected/2e67de1d-ae77-4772-beef-f28ec64b1647-kube-api-access-psjf4\") pod \"2e67de1d-ae77-4772-beef-f28ec64b1647\" (UID: \"2e67de1d-ae77-4772-beef-f28ec64b1647\") " Sep 09 21:27:48.951735 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.951712 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e67de1d-ae77-4772-beef-f28ec64b1647-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2e67de1d-ae77-4772-beef-f28ec64b1647" (UID: "2e67de1d-ae77-4772-beef-f28ec64b1647"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:27:48.953646 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.953622 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6-kube-api-access-jg5s7" (OuterVolumeSpecName: "kube-api-access-jg5s7") pod "fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" (UID: "fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6"). InnerVolumeSpecName "kube-api-access-jg5s7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:27:48.953758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:48.953656 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e67de1d-ae77-4772-beef-f28ec64b1647-kube-api-access-psjf4" (OuterVolumeSpecName: "kube-api-access-psjf4") pod "2e67de1d-ae77-4772-beef-f28ec64b1647" (UID: "2e67de1d-ae77-4772-beef-f28ec64b1647"). InnerVolumeSpecName "kube-api-access-psjf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:27:49.015587 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.015550 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" Sep 09 21:27:49.022710 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.022684 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" Sep 09 21:27:49.029695 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.029671 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp" Sep 09 21:27:49.053426 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.053390 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jg5s7\" (UniqueName: \"kubernetes.io/projected/fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6-kube-api-access-jg5s7\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:27:49.053426 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.053429 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psjf4\" (UniqueName: \"kubernetes.io/projected/2e67de1d-ae77-4772-beef-f28ec64b1647-kube-api-access-psjf4\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:27:49.053619 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.053446 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e67de1d-ae77-4772-beef-f28ec64b1647-extensions-socket-volume\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:27:49.327728 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.327703 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9"] Sep 09 21:27:49.328675 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:27:49.328643 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca8eeb0_1289_4179_9f43_54479fd61bb2.slice/crio-61f2ee978ec7de6dbe2019cacf3280b5999eb016c14e8e0b6d22d82ec7e18aa1 WatchSource:0}: Error finding container 61f2ee978ec7de6dbe2019cacf3280b5999eb016c14e8e0b6d22d82ec7e18aa1: Status 404 returned error can't find the container with id 61f2ee978ec7de6dbe2019cacf3280b5999eb016c14e8e0b6d22d82ec7e18aa1 Sep 09 21:27:49.332357 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.332337 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc"] Sep 09 21:27:49.334559 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:27:49.334541 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ce3699_b6cc_4239_b4a6_406a037de1d5.slice/crio-9ad4319d280289ec4cab5d57295408b3c0d180c4cfa010f0afb4b9f9d8b18051 WatchSource:0}: Error finding container 9ad4319d280289ec4cab5d57295408b3c0d180c4cfa010f0afb4b9f9d8b18051: Status 404 returned error can't find the container with id 9ad4319d280289ec4cab5d57295408b3c0d180c4cfa010f0afb4b9f9d8b18051 Sep 09 21:27:49.341970 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.341951 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp"] Sep 09 21:27:49.347149 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:27:49.347126 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23aad71c_05cb_4a06_a24c_764c9ea1cb1e.slice/crio-08ac92fa40ce96d00e2c28d3ebb25da6daf57f7e1d5fe834070f1ce558fc4ea1 WatchSource:0}: Error finding container 08ac92fa40ce96d00e2c28d3ebb25da6daf57f7e1d5fe834070f1ce558fc4ea1: Status 404 returned error can't find the container with id 08ac92fa40ce96d00e2c28d3ebb25da6daf57f7e1d5fe834070f1ce558fc4ea1 Sep 09 21:27:49.386451 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.385891 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" event={"ID":"dca8eeb0-1289-4179-9f43-54479fd61bb2","Type":"ContainerStarted","Data":"61f2ee978ec7de6dbe2019cacf3280b5999eb016c14e8e0b6d22d82ec7e18aa1"} Sep 09 21:27:49.387954 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.387929 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" event={"ID":"05ce3699-b6cc-4239-b4a6-406a037de1d5","Type":"ContainerStarted","Data":"9ad4319d280289ec4cab5d57295408b3c0d180c4cfa010f0afb4b9f9d8b18051"} Sep 09 21:27:49.389689 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.389591 2553 generic.go:356] "Generic (PLEG): container finished" podID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" containerID="a07a3b9525ef4e416c61b2048b145236f058b56083b3780ce15bf19509d700f3" exitCode=0 Sep 09 21:27:49.389689 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.389640 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:49.389877 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.389641 2553 scope.go:117] "RemoveContainer" containerID="a07a3b9525ef4e416c61b2048b145236f058b56083b3780ce15bf19509d700f3" Sep 09 21:27:49.392176 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.391717 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp" event={"ID":"23aad71c-05cb-4a06-a24c-764c9ea1cb1e","Type":"ContainerStarted","Data":"08ac92fa40ce96d00e2c28d3ebb25da6daf57f7e1d5fe834070f1ce558fc4ea1"} Sep 09 21:27:49.392615 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.392585 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-llb8l\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:49.394744 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.394708 2553 generic.go:356] "Generic (PLEG): container finished" podID="2e67de1d-ae77-4772-beef-f28ec64b1647" containerID="153ee0ba8d95574d5e1fe07e6ffb23ce072934a8e1588e798e695bd8d56b9ff1" exitCode=0 Sep 09 21:27:49.394744 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.394739 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:49.394981 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.394961 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:49.395700 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.395123 2553 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" container="manager" Sep 09 21:27:49.397368 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.397300 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:49.399566 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.399444 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-rfstt\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="a9ab342e-dd20-4aec-957c-516c4c9c98b7" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:49.400784 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.400761 2553 scope.go:117] "RemoveContainer" containerID="a07a3b9525ef4e416c61b2048b145236f058b56083b3780ce15bf19509d700f3" Sep 09 21:27:49.401337 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.401159 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a07a3b9525ef4e416c61b2048b145236f058b56083b3780ce15bf19509d700f3\": container with ID starting with a07a3b9525ef4e416c61b2048b145236f058b56083b3780ce15bf19509d700f3 not found: ID does not exist" containerID="a07a3b9525ef4e416c61b2048b145236f058b56083b3780ce15bf19509d700f3" Sep 09 21:27:49.401337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.401199 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a07a3b9525ef4e416c61b2048b145236f058b56083b3780ce15bf19509d700f3"} err="failed to get container status \"a07a3b9525ef4e416c61b2048b145236f058b56083b3780ce15bf19509d700f3\": rpc error: code = NotFound desc = could not find container \"a07a3b9525ef4e416c61b2048b145236f058b56083b3780ce15bf19509d700f3\": container with ID starting with a07a3b9525ef4e416c61b2048b145236f058b56083b3780ce15bf19509d700f3 not found: ID does not exist" Sep 09 21:27:49.401337 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.401222 2553 scope.go:117] "RemoveContainer" containerID="153ee0ba8d95574d5e1fe07e6ffb23ce072934a8e1588e798e695bd8d56b9ff1" Sep 09 21:27:49.401809 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.401630 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-llb8l\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:49.403953 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.403651 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:49.405996 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.405967 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-rfstt\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="a9ab342e-dd20-4aec-957c-516c4c9c98b7" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:49.408175 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.408145 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-llb8l\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:49.411921 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.411889 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:49.413592 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.413572 2553 scope.go:117] "RemoveContainer" containerID="153ee0ba8d95574d5e1fe07e6ffb23ce072934a8e1588e798e695bd8d56b9ff1" Sep 09 21:27:49.413937 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.413913 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"153ee0ba8d95574d5e1fe07e6ffb23ce072934a8e1588e798e695bd8d56b9ff1\": container with ID starting with 153ee0ba8d95574d5e1fe07e6ffb23ce072934a8e1588e798e695bd8d56b9ff1 not found: ID does not exist" containerID="153ee0ba8d95574d5e1fe07e6ffb23ce072934a8e1588e798e695bd8d56b9ff1" Sep 09 21:27:49.414031 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:49.413943 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"153ee0ba8d95574d5e1fe07e6ffb23ce072934a8e1588e798e695bd8d56b9ff1"} err="failed to get container status \"153ee0ba8d95574d5e1fe07e6ffb23ce072934a8e1588e798e695bd8d56b9ff1\": rpc error: code = NotFound desc = could not find container \"153ee0ba8d95574d5e1fe07e6ffb23ce072934a8e1588e798e695bd8d56b9ff1\": container with ID starting with 153ee0ba8d95574d5e1fe07e6ffb23ce072934a8e1588e798e695bd8d56b9ff1 not found: ID does not exist" Sep 09 21:27:49.414031 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.413954 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-rfstt\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="a9ab342e-dd20-4aec-957c-516c4c9c98b7" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:49.416082 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:49.416058 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-llb8l\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:50.389508 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:50.389461 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-vdggs\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-vdggs" Sep 09 21:27:50.391154 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.391130 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e67de1d-ae77-4772-beef-f28ec64b1647" path="/var/lib/kubelet/pods/2e67de1d-ae77-4772-beef-f28ec64b1647/volumes" Sep 09 21:27:50.391464 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.391451 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" path="/var/lib/kubelet/pods/fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6/volumes" Sep 09 21:27:50.391877 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:50.391856 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-rfstt\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="a9ab342e-dd20-4aec-957c-516c4c9c98b7" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:50.394744 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:50.394718 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-llb8l\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="fe4a7a0b-895b-4ed2-8a00-7a2a22c6beb6" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-llb8l" Sep 09 21:27:50.402034 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.402005 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp" event={"ID":"23aad71c-05cb-4a06-a24c-764c9ea1cb1e","Type":"ContainerStarted","Data":"9f6e5b648a43f81de6fb1c6694ddb6a939db94bec3c9c369d4c435d1b5b8982d"} Sep 09 21:27:50.402229 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.402118 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp" Sep 09 21:27:50.404751 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.404716 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" podUID="a9ab342e-dd20-4aec-957c-516c4c9c98b7" containerName="manager" containerID="cri-o://dd0492e222d5d6b957d8051b824fc31e23c55e6f4988c22bf0b87f875942679f" gracePeriod=2 Sep 09 21:27:50.406465 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.406439 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" event={"ID":"dca8eeb0-1289-4179-9f43-54479fd61bb2","Type":"ContainerStarted","Data":"b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1"} Sep 09 21:27:50.406584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.406518 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" Sep 09 21:27:50.408236 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.408213 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" event={"ID":"05ce3699-b6cc-4239-b4a6-406a037de1d5","Type":"ContainerStarted","Data":"a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d"} Sep 09 21:27:50.408358 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.408309 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" Sep 09 21:27:50.428017 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.427970 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp" podStartSLOduration=2.427955437 podStartE2EDuration="2.427955437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:27:50.425605079 +0000 UTC m=+1080.532348905" watchObservedRunningTime="2026-09-09 21:27:50.427955437 +0000 UTC m=+1080.534699262" Sep 09 21:27:50.448122 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.448041 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" podStartSLOduration=2.448020711 podStartE2EDuration="2.448020711s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:27:50.446272876 +0000 UTC m=+1080.553016703" watchObservedRunningTime="2026-09-09 21:27:50.448020711 +0000 UTC m=+1080.554764537" Sep 09 21:27:50.473986 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.473939 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" podStartSLOduration=2.473923392 podStartE2EDuration="2.473923392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:27:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:27:50.473492833 +0000 UTC m=+1080.580236680" watchObservedRunningTime="2026-09-09 21:27:50.473923392 +0000 UTC m=+1080.580667218" Sep 09 21:27:50.652370 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.652301 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:50.768878 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.768844 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9ab342e-dd20-4aec-957c-516c4c9c98b7-extensions-socket-volume\") pod \"a9ab342e-dd20-4aec-957c-516c4c9c98b7\" (UID: \"a9ab342e-dd20-4aec-957c-516c4c9c98b7\") " Sep 09 21:27:50.769059 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.768929 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7d8hz\" (UniqueName: \"kubernetes.io/projected/a9ab342e-dd20-4aec-957c-516c4c9c98b7-kube-api-access-7d8hz\") pod \"a9ab342e-dd20-4aec-957c-516c4c9c98b7\" (UID: \"a9ab342e-dd20-4aec-957c-516c4c9c98b7\") " Sep 09 21:27:50.769329 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.769304 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9ab342e-dd20-4aec-957c-516c4c9c98b7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a9ab342e-dd20-4aec-957c-516c4c9c98b7" (UID: "a9ab342e-dd20-4aec-957c-516c4c9c98b7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:27:50.770978 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.770953 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9ab342e-dd20-4aec-957c-516c4c9c98b7-kube-api-access-7d8hz" (OuterVolumeSpecName: "kube-api-access-7d8hz") pod "a9ab342e-dd20-4aec-957c-516c4c9c98b7" (UID: "a9ab342e-dd20-4aec-957c-516c4c9c98b7"). InnerVolumeSpecName "kube-api-access-7d8hz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:27:50.870085 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.870042 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9ab342e-dd20-4aec-957c-516c4c9c98b7-extensions-socket-volume\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:27:50.870085 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:50.870081 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7d8hz\" (UniqueName: \"kubernetes.io/projected/a9ab342e-dd20-4aec-957c-516c4c9c98b7-kube-api-access-7d8hz\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:27:51.416524 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:51.416490 2553 generic.go:356] "Generic (PLEG): container finished" podID="a9ab342e-dd20-4aec-957c-516c4c9c98b7" containerID="dd0492e222d5d6b957d8051b824fc31e23c55e6f4988c22bf0b87f875942679f" exitCode=0 Sep 09 21:27:51.416974 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:51.416531 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:51.416974 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:51.416574 2553 scope.go:117] "RemoveContainer" containerID="dd0492e222d5d6b957d8051b824fc31e23c55e6f4988c22bf0b87f875942679f" Sep 09 21:27:51.418960 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:51.418918 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-rfstt\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="a9ab342e-dd20-4aec-957c-516c4c9c98b7" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:51.425567 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:51.425548 2553 scope.go:117] "RemoveContainer" containerID="dd0492e222d5d6b957d8051b824fc31e23c55e6f4988c22bf0b87f875942679f" Sep 09 21:27:51.425865 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:51.425844 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd0492e222d5d6b957d8051b824fc31e23c55e6f4988c22bf0b87f875942679f\": container with ID starting with dd0492e222d5d6b957d8051b824fc31e23c55e6f4988c22bf0b87f875942679f not found: ID does not exist" containerID="dd0492e222d5d6b957d8051b824fc31e23c55e6f4988c22bf0b87f875942679f" Sep 09 21:27:51.425918 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:51.425874 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd0492e222d5d6b957d8051b824fc31e23c55e6f4988c22bf0b87f875942679f"} err="failed to get container status \"dd0492e222d5d6b957d8051b824fc31e23c55e6f4988c22bf0b87f875942679f\": rpc error: code = NotFound desc = could not find container \"dd0492e222d5d6b957d8051b824fc31e23c55e6f4988c22bf0b87f875942679f\": container with ID starting with dd0492e222d5d6b957d8051b824fc31e23c55e6f4988c22bf0b87f875942679f not found: ID does not exist" Sep 09 21:27:51.427467 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:27:51.427435 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-rfstt\" is forbidden: User \"system:node:ip-10-0-137-115.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-115.ec2.internal' and this object" podUID="a9ab342e-dd20-4aec-957c-516c4c9c98b7" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-rfstt" Sep 09 21:27:52.389698 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:27:52.389665 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9ab342e-dd20-4aec-957c-516c4c9c98b7" path="/var/lib/kubelet/pods/a9ab342e-dd20-4aec-957c-516c4c9c98b7/volumes" Sep 09 21:28:01.419252 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.419220 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-wgfgp" Sep 09 21:28:01.419766 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.419289 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" Sep 09 21:28:01.419766 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.419321 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" Sep 09 21:28:01.477059 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.477024 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9"] Sep 09 21:28:01.477283 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.477258 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" podUID="dca8eeb0-1289-4179-9f43-54479fd61bb2" containerName="manager" containerID="cri-o://b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1" gracePeriod=10 Sep 09 21:28:01.698913 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.698872 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2"] Sep 09 21:28:01.703020 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.702999 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" Sep 09 21:28:01.721141 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.720945 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2"] Sep 09 21:28:01.738996 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.738973 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" Sep 09 21:28:01.772506 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.772471 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4mp9\" (UniqueName: \"kubernetes.io/projected/db31f506-8059-4d16-bb7e-98adb21a9350-kube-api-access-b4mp9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7mwx2\" (UID: \"db31f506-8059-4d16-bb7e-98adb21a9350\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" Sep 09 21:28:01.772712 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.772512 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db31f506-8059-4d16-bb7e-98adb21a9350-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7mwx2\" (UID: \"db31f506-8059-4d16-bb7e-98adb21a9350\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" Sep 09 21:28:01.873138 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.873083 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5sf7\" (UniqueName: \"kubernetes.io/projected/dca8eeb0-1289-4179-9f43-54479fd61bb2-kube-api-access-d5sf7\") pod \"dca8eeb0-1289-4179-9f43-54479fd61bb2\" (UID: \"dca8eeb0-1289-4179-9f43-54479fd61bb2\") " Sep 09 21:28:01.873340 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.873154 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dca8eeb0-1289-4179-9f43-54479fd61bb2-extensions-socket-volume\") pod \"dca8eeb0-1289-4179-9f43-54479fd61bb2\" (UID: \"dca8eeb0-1289-4179-9f43-54479fd61bb2\") " Sep 09 21:28:01.873340 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.873280 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db31f506-8059-4d16-bb7e-98adb21a9350-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7mwx2\" (UID: \"db31f506-8059-4d16-bb7e-98adb21a9350\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" Sep 09 21:28:01.873466 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.873384 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4mp9\" (UniqueName: \"kubernetes.io/projected/db31f506-8059-4d16-bb7e-98adb21a9350-kube-api-access-b4mp9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7mwx2\" (UID: \"db31f506-8059-4d16-bb7e-98adb21a9350\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" Sep 09 21:28:01.873616 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.873590 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dca8eeb0-1289-4179-9f43-54479fd61bb2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dca8eeb0-1289-4179-9f43-54479fd61bb2" (UID: "dca8eeb0-1289-4179-9f43-54479fd61bb2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:28:01.873697 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.873681 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db31f506-8059-4d16-bb7e-98adb21a9350-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7mwx2\" (UID: \"db31f506-8059-4d16-bb7e-98adb21a9350\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" Sep 09 21:28:01.875272 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.875249 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dca8eeb0-1289-4179-9f43-54479fd61bb2-kube-api-access-d5sf7" (OuterVolumeSpecName: "kube-api-access-d5sf7") pod "dca8eeb0-1289-4179-9f43-54479fd61bb2" (UID: "dca8eeb0-1289-4179-9f43-54479fd61bb2"). InnerVolumeSpecName "kube-api-access-d5sf7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:28:01.882222 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.882198 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4mp9\" (UniqueName: \"kubernetes.io/projected/db31f506-8059-4d16-bb7e-98adb21a9350-kube-api-access-b4mp9\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7mwx2\" (UID: \"db31f506-8059-4d16-bb7e-98adb21a9350\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" Sep 09 21:28:01.973915 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.973830 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5sf7\" (UniqueName: \"kubernetes.io/projected/dca8eeb0-1289-4179-9f43-54479fd61bb2-kube-api-access-d5sf7\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:28:01.973915 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:01.973858 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dca8eeb0-1289-4179-9f43-54479fd61bb2-extensions-socket-volume\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:28:02.014336 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.014308 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" Sep 09 21:28:02.296558 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.296533 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2"] Sep 09 21:28:02.299044 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:28:02.299021 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb31f506_8059_4d16_bb7e_98adb21a9350.slice/crio-b771de2196d1bb05b36e92bd71d04d1b958fc1db027e8c0ec67e47d889ca6d7b WatchSource:0}: Error finding container b771de2196d1bb05b36e92bd71d04d1b958fc1db027e8c0ec67e47d889ca6d7b: Status 404 returned error can't find the container with id b771de2196d1bb05b36e92bd71d04d1b958fc1db027e8c0ec67e47d889ca6d7b Sep 09 21:28:02.479155 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.479117 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" event={"ID":"db31f506-8059-4d16-bb7e-98adb21a9350","Type":"ContainerStarted","Data":"0596450098ffb97e9e57c3cc6257ea03bc284d0cf5a6cb9fcb295af69d2dc020"} Sep 09 21:28:02.479631 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.479186 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" Sep 09 21:28:02.479631 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.479210 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" event={"ID":"db31f506-8059-4d16-bb7e-98adb21a9350","Type":"ContainerStarted","Data":"b771de2196d1bb05b36e92bd71d04d1b958fc1db027e8c0ec67e47d889ca6d7b"} Sep 09 21:28:02.480693 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.480667 2553 generic.go:356] "Generic (PLEG): container finished" podID="dca8eeb0-1289-4179-9f43-54479fd61bb2" containerID="b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1" exitCode=0 Sep 09 21:28:02.480774 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.480700 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" Sep 09 21:28:02.480774 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.480739 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" event={"ID":"dca8eeb0-1289-4179-9f43-54479fd61bb2","Type":"ContainerDied","Data":"b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1"} Sep 09 21:28:02.480860 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.480787 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9" event={"ID":"dca8eeb0-1289-4179-9f43-54479fd61bb2","Type":"ContainerDied","Data":"61f2ee978ec7de6dbe2019cacf3280b5999eb016c14e8e0b6d22d82ec7e18aa1"} Sep 09 21:28:02.480860 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.480809 2553 scope.go:117] "RemoveContainer" containerID="b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1" Sep 09 21:28:02.489290 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.489272 2553 scope.go:117] "RemoveContainer" containerID="b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1" Sep 09 21:28:02.489587 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:28:02.489566 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1\": container with ID starting with b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1 not found: ID does not exist" containerID="b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1" Sep 09 21:28:02.489650 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.489601 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1"} err="failed to get container status \"b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1\": rpc error: code = NotFound desc = could not find container \"b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1\": container with ID starting with b0ca4bb5effda0363aa1b8f1146033c61c7ab347319d277b7f4ff1d820de8bf1 not found: ID does not exist" Sep 09 21:28:02.512776 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.512719 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" podStartSLOduration=1.5127009390000001 podStartE2EDuration="1.512700939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:28:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:28:02.511073017 +0000 UTC m=+1092.617816853" watchObservedRunningTime="2026-09-09 21:28:02.512700939 +0000 UTC m=+1092.619444765" Sep 09 21:28:02.539067 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.539037 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9"] Sep 09 21:28:02.554182 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:02.552273 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nbx9"] Sep 09 21:28:04.389579 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:04.389549 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dca8eeb0-1289-4179-9f43-54479fd61bb2" path="/var/lib/kubelet/pods/dca8eeb0-1289-4179-9f43-54479fd61bb2/volumes" Sep 09 21:28:13.489598 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:13.489566 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7mwx2" Sep 09 21:28:13.547542 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:13.547511 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc"] Sep 09 21:28:13.547954 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:13.547920 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" podUID="05ce3699-b6cc-4239-b4a6-406a037de1d5" containerName="manager" containerID="cri-o://a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d" gracePeriod=10 Sep 09 21:28:13.803988 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:13.803963 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" Sep 09 21:28:13.981986 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:13.981960 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05ce3699-b6cc-4239-b4a6-406a037de1d5-extensions-socket-volume\") pod \"05ce3699-b6cc-4239-b4a6-406a037de1d5\" (UID: \"05ce3699-b6cc-4239-b4a6-406a037de1d5\") " Sep 09 21:28:13.982160 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:13.982047 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4fq4\" (UniqueName: \"kubernetes.io/projected/05ce3699-b6cc-4239-b4a6-406a037de1d5-kube-api-access-w4fq4\") pod \"05ce3699-b6cc-4239-b4a6-406a037de1d5\" (UID: \"05ce3699-b6cc-4239-b4a6-406a037de1d5\") " Sep 09 21:28:13.982329 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:13.982300 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05ce3699-b6cc-4239-b4a6-406a037de1d5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "05ce3699-b6cc-4239-b4a6-406a037de1d5" (UID: "05ce3699-b6cc-4239-b4a6-406a037de1d5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Sep 09 21:28:13.984069 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:13.984045 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05ce3699-b6cc-4239-b4a6-406a037de1d5-kube-api-access-w4fq4" (OuterVolumeSpecName: "kube-api-access-w4fq4") pod "05ce3699-b6cc-4239-b4a6-406a037de1d5" (UID: "05ce3699-b6cc-4239-b4a6-406a037de1d5"). InnerVolumeSpecName "kube-api-access-w4fq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:28:14.083160 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:14.083075 2553 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05ce3699-b6cc-4239-b4a6-406a037de1d5-extensions-socket-volume\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:28:14.083160 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:14.083129 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4fq4\" (UniqueName: \"kubernetes.io/projected/05ce3699-b6cc-4239-b4a6-406a037de1d5-kube-api-access-w4fq4\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:28:14.544893 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:14.544854 2553 generic.go:356] "Generic (PLEG): container finished" podID="05ce3699-b6cc-4239-b4a6-406a037de1d5" containerID="a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d" exitCode=0 Sep 09 21:28:14.545334 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:14.544901 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" Sep 09 21:28:14.545334 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:14.544936 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" event={"ID":"05ce3699-b6cc-4239-b4a6-406a037de1d5","Type":"ContainerDied","Data":"a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d"} Sep 09 21:28:14.545334 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:14.544987 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc" event={"ID":"05ce3699-b6cc-4239-b4a6-406a037de1d5","Type":"ContainerDied","Data":"9ad4319d280289ec4cab5d57295408b3c0d180c4cfa010f0afb4b9f9d8b18051"} Sep 09 21:28:14.545334 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:14.545008 2553 scope.go:117] "RemoveContainer" containerID="a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d" Sep 09 21:28:14.554020 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:14.553826 2553 scope.go:117] "RemoveContainer" containerID="a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d" Sep 09 21:28:14.554157 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:28:14.554135 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d\": container with ID starting with a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d not found: ID does not exist" containerID="a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d" Sep 09 21:28:14.554213 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:14.554165 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d"} err="failed to get container status \"a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d\": rpc error: code = NotFound desc = could not find container \"a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d\": container with ID starting with a2ea5aae26aa65fd16efee6f3337e169909687f96602a6d3e0934f6925e07e4d not found: ID does not exist" Sep 09 21:28:14.565617 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:14.565594 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc"] Sep 09 21:28:14.569710 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:14.569689 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-g6kfc"] Sep 09 21:28:16.389142 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:16.389080 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05ce3699-b6cc-4239-b4a6-406a037de1d5" path="/var/lib/kubelet/pods/05ce3699-b6cc-4239-b4a6-406a037de1d5/volumes" Sep 09 21:28:17.850478 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:17.850448 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6"] Sep 09 21:28:17.850854 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:17.850835 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dca8eeb0-1289-4179-9f43-54479fd61bb2" containerName="manager" Sep 09 21:28:17.850854 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:17.850845 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="dca8eeb0-1289-4179-9f43-54479fd61bb2" containerName="manager" Sep 09 21:28:17.850938 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:17.850863 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05ce3699-b6cc-4239-b4a6-406a037de1d5" containerName="manager" Sep 09 21:28:17.850938 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:17.850868 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="05ce3699-b6cc-4239-b4a6-406a037de1d5" containerName="manager" Sep 09 21:28:17.851008 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:17.850938 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="05ce3699-b6cc-4239-b4a6-406a037de1d5" containerName="manager" Sep 09 21:28:17.851008 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:17.850948 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="dca8eeb0-1289-4179-9f43-54479fd61bb2" containerName="manager" Sep 09 21:28:17.853902 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:17.853886 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:17.856726 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:17.856706 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-pp8s5\"" Sep 09 21:28:17.856844 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:17.856710 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Sep 09 21:28:17.864558 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:17.864537 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6"] Sep 09 21:28:18.020006 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.019975 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.020223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.020014 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.020223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.020056 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.020223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.020123 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.020223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.020150 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.020223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.020175 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9n5w\" (UniqueName: \"kubernetes.io/projected/3a4850ff-3bce-4850-abd4-17ae634f1fa3-kube-api-access-h9n5w\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.020223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.020196 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.020439 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.020237 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.020439 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.020270 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.120931 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.120834 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.120931 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.120879 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.120931 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.120897 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.121224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.120935 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.121224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.120967 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.121224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.121027 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.121224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.121077 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.121224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.121135 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.121224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.121178 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9n5w\" (UniqueName: \"kubernetes.io/projected/3a4850ff-3bce-4850-abd4-17ae634f1fa3-kube-api-access-h9n5w\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.121521 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.121355 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.121521 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.121409 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.121521 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.121426 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.121690 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.121599 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.121690 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.121621 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.123470 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.123449 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.123719 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.123694 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.131020 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.130993 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a4850ff-3bce-4850-abd4-17ae634f1fa3-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.131157 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.131135 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9n5w\" (UniqueName: \"kubernetes.io/projected/3a4850ff-3bce-4850-abd4-17ae634f1fa3-kube-api-access-h9n5w\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hplk6\" (UID: \"3a4850ff-3bce-4850-abd4-17ae634f1fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.165138 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.165088 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:18.440869 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.440844 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6"] Sep 09 21:28:18.442597 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:28:18.442569 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a4850ff_3bce_4850_abd4_17ae634f1fa3.slice/crio-826df41d2aa9fefe723ddfcf2c2fc1ac9bddbe199000d63e127381d010a1f6df WatchSource:0}: Error finding container 826df41d2aa9fefe723ddfcf2c2fc1ac9bddbe199000d63e127381d010a1f6df: Status 404 returned error can't find the container with id 826df41d2aa9fefe723ddfcf2c2fc1ac9bddbe199000d63e127381d010a1f6df Sep 09 21:28:18.567908 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:18.567874 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" event={"ID":"3a4850ff-3bce-4850-abd4-17ae634f1fa3","Type":"ContainerStarted","Data":"826df41d2aa9fefe723ddfcf2c2fc1ac9bddbe199000d63e127381d010a1f6df"} Sep 09 21:28:21.145796 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:21.145757 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 21:28:21.146165 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:21.145850 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 21:28:21.146165 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:21.145900 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 21:28:21.586523 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:21.586489 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" event={"ID":"3a4850ff-3bce-4850-abd4-17ae634f1fa3","Type":"ContainerStarted","Data":"95f1306c366e7e4e037746af7aa88a32261e9954df903d77dd643f1739f9b881"} Sep 09 21:28:21.614527 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:21.609885 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" podStartSLOduration=1.908980983 podStartE2EDuration="4.60986253s" totalImagesPullingTime="2.700881547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:28:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:28:21.60697784 +0000 UTC m=+1111.713721667" watchObservedRunningTime="2026-09-09 21:28:21.60986253 +0000 UTC m=+1111.716606357" Sep 09 21:28:22.165771 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:22.165732 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:22.170764 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:22.170734 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:22.592751 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:22.592709 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:22.593696 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:22.593677 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:28:33.988711 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:33.988658 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5x2m6"] Sep 09 21:28:33.993636 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:33.993613 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" Sep 09 21:28:33.996520 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:33.996495 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Sep 09 21:28:33.996661 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:33.996554 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dl44v\"" Sep 09 21:28:34.000773 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:34.000747 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5x2m6"] Sep 09 21:28:34.059493 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:34.059462 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/156f7f02-0ddc-4b90-9691-34c557de7288-config-file\") pod \"limitador-limitador-8588749977-5x2m6\" (UID: \"156f7f02-0ddc-4b90-9691-34c557de7288\") " pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" Sep 09 21:28:34.059493 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:34.059508 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6wl6\" (UniqueName: \"kubernetes.io/projected/156f7f02-0ddc-4b90-9691-34c557de7288-kube-api-access-p6wl6\") pod \"limitador-limitador-8588749977-5x2m6\" (UID: \"156f7f02-0ddc-4b90-9691-34c557de7288\") " pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" Sep 09 21:28:34.087700 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:34.087666 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5x2m6"] Sep 09 21:28:34.161115 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:34.161072 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/156f7f02-0ddc-4b90-9691-34c557de7288-config-file\") pod \"limitador-limitador-8588749977-5x2m6\" (UID: \"156f7f02-0ddc-4b90-9691-34c557de7288\") " pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" Sep 09 21:28:34.161283 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:34.161165 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6wl6\" (UniqueName: \"kubernetes.io/projected/156f7f02-0ddc-4b90-9691-34c557de7288-kube-api-access-p6wl6\") pod \"limitador-limitador-8588749977-5x2m6\" (UID: \"156f7f02-0ddc-4b90-9691-34c557de7288\") " pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" Sep 09 21:28:34.161743 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:34.161726 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/156f7f02-0ddc-4b90-9691-34c557de7288-config-file\") pod \"limitador-limitador-8588749977-5x2m6\" (UID: \"156f7f02-0ddc-4b90-9691-34c557de7288\") " pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" Sep 09 21:28:34.170567 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:34.170534 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6wl6\" (UniqueName: \"kubernetes.io/projected/156f7f02-0ddc-4b90-9691-34c557de7288-kube-api-access-p6wl6\") pod \"limitador-limitador-8588749977-5x2m6\" (UID: \"156f7f02-0ddc-4b90-9691-34c557de7288\") " pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" Sep 09 21:28:34.304770 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:34.304652 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" Sep 09 21:28:34.576678 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:34.576653 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5x2m6"] Sep 09 21:28:34.578629 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:28:34.578604 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod156f7f02_0ddc_4b90_9691_34c557de7288.slice/crio-5bf2a6d807182fa0fe49c485ab8c680a0c5251de9a35a814fcc4b3adbb917735 WatchSource:0}: Error finding container 5bf2a6d807182fa0fe49c485ab8c680a0c5251de9a35a814fcc4b3adbb917735: Status 404 returned error can't find the container with id 5bf2a6d807182fa0fe49c485ab8c680a0c5251de9a35a814fcc4b3adbb917735 Sep 09 21:28:34.658725 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:34.658692 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" event={"ID":"156f7f02-0ddc-4b90-9691-34c557de7288","Type":"ContainerStarted","Data":"5bf2a6d807182fa0fe49c485ab8c680a0c5251de9a35a814fcc4b3adbb917735"} Sep 09 21:28:36.672210 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:36.672170 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" event={"ID":"156f7f02-0ddc-4b90-9691-34c557de7288","Type":"ContainerStarted","Data":"b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130"} Sep 09 21:28:36.672651 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:36.672228 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" Sep 09 21:28:36.690458 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:36.690411 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" podStartSLOduration=2.146938448 podStartE2EDuration="3.690397041s" totalImagesPullingTime="1.543458593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:28:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:28:36.687922005 +0000 UTC m=+1126.794665851" watchObservedRunningTime="2026-09-09 21:28:36.690397041 +0000 UTC m=+1126.797140865" Sep 09 21:28:47.678247 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:47.678211 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" Sep 09 21:28:50.378385 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:50.378334 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5x2m6"] Sep 09 21:28:50.378804 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:50.378599 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" podUID="156f7f02-0ddc-4b90-9691-34c557de7288" containerName="limitador" containerID="cri-o://b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130" gracePeriod=30 Sep 09 21:28:50.944058 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:50.943977 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" Sep 09 21:28:51.014275 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.014243 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6wl6\" (UniqueName: \"kubernetes.io/projected/156f7f02-0ddc-4b90-9691-34c557de7288-kube-api-access-p6wl6\") pod \"156f7f02-0ddc-4b90-9691-34c557de7288\" (UID: \"156f7f02-0ddc-4b90-9691-34c557de7288\") " Sep 09 21:28:51.014461 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.014295 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/156f7f02-0ddc-4b90-9691-34c557de7288-config-file\") pod \"156f7f02-0ddc-4b90-9691-34c557de7288\" (UID: \"156f7f02-0ddc-4b90-9691-34c557de7288\") " Sep 09 21:28:51.014637 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.014612 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/156f7f02-0ddc-4b90-9691-34c557de7288-config-file" (OuterVolumeSpecName: "config-file") pod "156f7f02-0ddc-4b90-9691-34c557de7288" (UID: "156f7f02-0ddc-4b90-9691-34c557de7288"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Sep 09 21:28:51.016281 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.016256 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/156f7f02-0ddc-4b90-9691-34c557de7288-kube-api-access-p6wl6" (OuterVolumeSpecName: "kube-api-access-p6wl6") pod "156f7f02-0ddc-4b90-9691-34c557de7288" (UID: "156f7f02-0ddc-4b90-9691-34c557de7288"). InnerVolumeSpecName "kube-api-access-p6wl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:28:51.115466 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.115413 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6wl6\" (UniqueName: \"kubernetes.io/projected/156f7f02-0ddc-4b90-9691-34c557de7288-kube-api-access-p6wl6\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:28:51.115466 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.115458 2553 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/156f7f02-0ddc-4b90-9691-34c557de7288-config-file\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:28:51.751037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.751000 2553 generic.go:356] "Generic (PLEG): container finished" podID="156f7f02-0ddc-4b90-9691-34c557de7288" containerID="b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130" exitCode=0 Sep 09 21:28:51.751037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.751039 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" Sep 09 21:28:51.751513 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.751082 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" event={"ID":"156f7f02-0ddc-4b90-9691-34c557de7288","Type":"ContainerDied","Data":"b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130"} Sep 09 21:28:51.751513 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.751151 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-5x2m6" event={"ID":"156f7f02-0ddc-4b90-9691-34c557de7288","Type":"ContainerDied","Data":"5bf2a6d807182fa0fe49c485ab8c680a0c5251de9a35a814fcc4b3adbb917735"} Sep 09 21:28:51.751513 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.751173 2553 scope.go:117] "RemoveContainer" containerID="b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130" Sep 09 21:28:51.761167 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.761143 2553 scope.go:117] "RemoveContainer" containerID="b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130" Sep 09 21:28:51.761493 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:28:51.761471 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130\": container with ID starting with b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130 not found: ID does not exist" containerID="b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130" Sep 09 21:28:51.761571 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.761501 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130"} err="failed to get container status \"b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130\": rpc error: code = NotFound desc = could not find container \"b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130\": container with ID starting with b86a5f1e40d33b36d294b828aaa61dfd7a5c152d94df6e1f511c61c20a1a5130 not found: ID does not exist" Sep 09 21:28:51.775499 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.775470 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5x2m6"] Sep 09 21:28:51.779202 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:51.779178 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-5x2m6"] Sep 09 21:28:52.390325 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:52.390293 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="156f7f02-0ddc-4b90-9691-34c557de7288" path="/var/lib/kubelet/pods/156f7f02-0ddc-4b90-9691-34c557de7288/volumes" Sep 09 21:28:54.465069 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.465029 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5x52n"] Sep 09 21:28:54.465701 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.465680 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="156f7f02-0ddc-4b90-9691-34c557de7288" containerName="limitador" Sep 09 21:28:54.465796 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.465705 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="156f7f02-0ddc-4b90-9691-34c557de7288" containerName="limitador" Sep 09 21:28:54.465852 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.465812 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="156f7f02-0ddc-4b90-9691-34c557de7288" containerName="limitador" Sep 09 21:28:54.473802 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.473778 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" Sep 09 21:28:54.476665 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.476630 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Sep 09 21:28:54.476858 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.476641 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Sep 09 21:28:54.476978 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.476730 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-rv5nm\"" Sep 09 21:28:54.476978 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.476760 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Sep 09 21:28:54.480935 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.480913 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5x52n"] Sep 09 21:28:54.546392 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.546357 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/331b37ea-9053-4a2b-b1ff-973775eacc06-data\") pod \"postgres-69fb4594cf-5x52n\" (UID: \"331b37ea-9053-4a2b-b1ff-973775eacc06\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" Sep 09 21:28:54.546583 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.546428 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k6zk\" (UniqueName: \"kubernetes.io/projected/331b37ea-9053-4a2b-b1ff-973775eacc06-kube-api-access-7k6zk\") pod \"postgres-69fb4594cf-5x52n\" (UID: \"331b37ea-9053-4a2b-b1ff-973775eacc06\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" Sep 09 21:28:54.646967 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.646924 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k6zk\" (UniqueName: \"kubernetes.io/projected/331b37ea-9053-4a2b-b1ff-973775eacc06-kube-api-access-7k6zk\") pod \"postgres-69fb4594cf-5x52n\" (UID: \"331b37ea-9053-4a2b-b1ff-973775eacc06\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" Sep 09 21:28:54.647180 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.647018 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/331b37ea-9053-4a2b-b1ff-973775eacc06-data\") pod \"postgres-69fb4594cf-5x52n\" (UID: \"331b37ea-9053-4a2b-b1ff-973775eacc06\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" Sep 09 21:28:54.647383 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.647366 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/331b37ea-9053-4a2b-b1ff-973775eacc06-data\") pod \"postgres-69fb4594cf-5x52n\" (UID: \"331b37ea-9053-4a2b-b1ff-973775eacc06\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" Sep 09 21:28:54.655530 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.655509 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k6zk\" (UniqueName: \"kubernetes.io/projected/331b37ea-9053-4a2b-b1ff-973775eacc06-kube-api-access-7k6zk\") pod \"postgres-69fb4594cf-5x52n\" (UID: \"331b37ea-9053-4a2b-b1ff-973775eacc06\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" Sep 09 21:28:54.786726 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:54.786634 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" Sep 09 21:28:55.062929 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:55.062903 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-5x52n"] Sep 09 21:28:55.064526 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:28:55.064499 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod331b37ea_9053_4a2b_b1ff_973775eacc06.slice/crio-52e34d82a3c3252fadfcef3d4e6a11dbf1c6aa4af76ac8c26ee69b6a3bd82702 WatchSource:0}: Error finding container 52e34d82a3c3252fadfcef3d4e6a11dbf1c6aa4af76ac8c26ee69b6a3bd82702: Status 404 returned error can't find the container with id 52e34d82a3c3252fadfcef3d4e6a11dbf1c6aa4af76ac8c26ee69b6a3bd82702 Sep 09 21:28:55.773988 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:28:55.773946 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" event={"ID":"331b37ea-9053-4a2b-b1ff-973775eacc06","Type":"ContainerStarted","Data":"52e34d82a3c3252fadfcef3d4e6a11dbf1c6aa4af76ac8c26ee69b6a3bd82702"} Sep 09 21:29:00.805760 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:00.805717 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" event={"ID":"331b37ea-9053-4a2b-b1ff-973775eacc06","Type":"ContainerStarted","Data":"33da82b213ba9afd2a620f8f7e18cf87bd2aacbd131bdeeb73b90397f9710dbf"} Sep 09 21:29:00.806231 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:00.805916 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" Sep 09 21:29:00.823003 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:00.822948 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" podStartSLOduration=1.639059562 podStartE2EDuration="6.822932239s" totalImagesPullingTime="5.183872677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:28:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:29:00.821906945 +0000 UTC m=+1150.928650790" watchObservedRunningTime="2026-09-09 21:29:00.822932239 +0000 UTC m=+1150.929676064" Sep 09 21:29:06.840423 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:06.840394 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-5x52n" Sep 09 21:29:14.137679 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.137598 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-587659f668-k4jn6"] Sep 09 21:29:14.143338 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.143319 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:14.146083 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.146057 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Sep 09 21:29:14.146224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.146123 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xfn58\"" Sep 09 21:29:14.146224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.146124 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Sep 09 21:29:14.146357 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.146229 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Sep 09 21:29:14.152553 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.152529 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-587659f668-k4jn6"] Sep 09 21:29:14.230084 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.230046 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-metrics-certs\") pod \"maas-controller-587659f668-k4jn6\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:14.230281 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.230145 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-webhook-cert\") pod \"maas-controller-587659f668-k4jn6\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:14.230281 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.230191 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw52r\" (UniqueName: \"kubernetes.io/projected/a770cafc-9c95-4517-9792-3796286cdfad-kube-api-access-vw52r\") pod \"maas-controller-587659f668-k4jn6\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:14.320420 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.320392 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-88ddf69cd-w4qmv"] Sep 09 21:29:14.324540 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.324516 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:14.331167 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.331052 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw52r\" (UniqueName: \"kubernetes.io/projected/a770cafc-9c95-4517-9792-3796286cdfad-kube-api-access-vw52r\") pod \"maas-controller-587659f668-k4jn6\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:14.331255 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.331230 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-metrics-certs\") pod \"maas-controller-587659f668-k4jn6\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:14.331746 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.331337 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-webhook-cert\") pod \"maas-controller-587659f668-k4jn6\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:14.331746 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:29:14.331453 2553 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 09 21:29:14.331746 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:29:14.331510 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-webhook-cert podName:a770cafc-9c95-4517-9792-3796286cdfad nodeName:}" failed. No retries permitted until 2026-09-09 21:29:14.831491483 +0000 UTC m=+1164.938235300 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-webhook-cert") pod "maas-controller-587659f668-k4jn6" (UID: "a770cafc-9c95-4517-9792-3796286cdfad") : secret "maas-controller-webhook-cert" not found Sep 09 21:29:14.331746 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:29:14.331511 2553 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found Sep 09 21:29:14.331746 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:29:14.331569 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-metrics-certs podName:a770cafc-9c95-4517-9792-3796286cdfad nodeName:}" failed. No retries permitted until 2026-09-09 21:29:14.831550486 +0000 UTC m=+1164.938294304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-metrics-certs") pod "maas-controller-587659f668-k4jn6" (UID: "a770cafc-9c95-4517-9792-3796286cdfad") : secret "maas-controller-metrics-tls" not found Sep 09 21:29:14.333284 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.333260 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-88ddf69cd-w4qmv"] Sep 09 21:29:14.339921 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.339891 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw52r\" (UniqueName: \"kubernetes.io/projected/a770cafc-9c95-4517-9792-3796286cdfad-kube-api-access-vw52r\") pod \"maas-controller-587659f668-k4jn6\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:14.432263 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.432234 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/badb67c5-7566-4353-8736-b79a3732266f-metrics-certs\") pod \"maas-controller-88ddf69cd-w4qmv\" (UID: \"badb67c5-7566-4353-8736-b79a3732266f\") " pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:14.432417 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.432291 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nn6lp\" (UniqueName: \"kubernetes.io/projected/badb67c5-7566-4353-8736-b79a3732266f-kube-api-access-nn6lp\") pod \"maas-controller-88ddf69cd-w4qmv\" (UID: \"badb67c5-7566-4353-8736-b79a3732266f\") " pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:14.432417 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.432332 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/badb67c5-7566-4353-8736-b79a3732266f-webhook-cert\") pod \"maas-controller-88ddf69cd-w4qmv\" (UID: \"badb67c5-7566-4353-8736-b79a3732266f\") " pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:14.533184 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.533149 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/badb67c5-7566-4353-8736-b79a3732266f-metrics-certs\") pod \"maas-controller-88ddf69cd-w4qmv\" (UID: \"badb67c5-7566-4353-8736-b79a3732266f\") " pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:14.533355 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.533213 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nn6lp\" (UniqueName: \"kubernetes.io/projected/badb67c5-7566-4353-8736-b79a3732266f-kube-api-access-nn6lp\") pod \"maas-controller-88ddf69cd-w4qmv\" (UID: \"badb67c5-7566-4353-8736-b79a3732266f\") " pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:14.533355 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.533240 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/badb67c5-7566-4353-8736-b79a3732266f-webhook-cert\") pod \"maas-controller-88ddf69cd-w4qmv\" (UID: \"badb67c5-7566-4353-8736-b79a3732266f\") " pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:14.533430 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:29:14.533351 2553 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Sep 09 21:29:14.533465 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:29:14.533429 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/badb67c5-7566-4353-8736-b79a3732266f-webhook-cert podName:badb67c5-7566-4353-8736-b79a3732266f nodeName:}" failed. No retries permitted until 2026-09-09 21:29:15.033411639 +0000 UTC m=+1165.140155447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/badb67c5-7566-4353-8736-b79a3732266f-webhook-cert") pod "maas-controller-88ddf69cd-w4qmv" (UID: "badb67c5-7566-4353-8736-b79a3732266f") : secret "maas-controller-webhook-cert" not found Sep 09 21:29:14.535695 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.535672 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/badb67c5-7566-4353-8736-b79a3732266f-metrics-certs\") pod \"maas-controller-88ddf69cd-w4qmv\" (UID: \"badb67c5-7566-4353-8736-b79a3732266f\") " pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:14.541627 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.541602 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nn6lp\" (UniqueName: \"kubernetes.io/projected/badb67c5-7566-4353-8736-b79a3732266f-kube-api-access-nn6lp\") pod \"maas-controller-88ddf69cd-w4qmv\" (UID: \"badb67c5-7566-4353-8736-b79a3732266f\") " pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:14.836452 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.836354 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-metrics-certs\") pod \"maas-controller-587659f668-k4jn6\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:14.836452 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.836421 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-webhook-cert\") pod \"maas-controller-587659f668-k4jn6\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:14.838833 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.838810 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-webhook-cert\") pod \"maas-controller-587659f668-k4jn6\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:14.838947 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:14.838853 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-metrics-certs\") pod \"maas-controller-587659f668-k4jn6\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:15.038401 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:15.038350 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/badb67c5-7566-4353-8736-b79a3732266f-webhook-cert\") pod \"maas-controller-88ddf69cd-w4qmv\" (UID: \"badb67c5-7566-4353-8736-b79a3732266f\") " pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:15.040730 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:15.040711 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/badb67c5-7566-4353-8736-b79a3732266f-webhook-cert\") pod \"maas-controller-88ddf69cd-w4qmv\" (UID: \"badb67c5-7566-4353-8736-b79a3732266f\") " pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:15.054791 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:15.054763 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:15.236815 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:15.236780 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:15.327233 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:15.327206 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-587659f668-k4jn6"] Sep 09 21:29:15.328678 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:29:15.328632 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda770cafc_9c95_4517_9792_3796286cdfad.slice/crio-186431a02a51f8b74464c8668fe43fda64c349f7db1287ac411faa0f26a2aa02 WatchSource:0}: Error finding container 186431a02a51f8b74464c8668fe43fda64c349f7db1287ac411faa0f26a2aa02: Status 404 returned error can't find the container with id 186431a02a51f8b74464c8668fe43fda64c349f7db1287ac411faa0f26a2aa02 Sep 09 21:29:15.508696 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:15.508669 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-88ddf69cd-w4qmv"] Sep 09 21:29:15.511323 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:29:15.511291 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbadb67c5_7566_4353_8736_b79a3732266f.slice/crio-465bcf0aa5fb22e5838145697eec67b9ae747d5a439d6ef77e300d4d560e5fea WatchSource:0}: Error finding container 465bcf0aa5fb22e5838145697eec67b9ae747d5a439d6ef77e300d4d560e5fea: Status 404 returned error can't find the container with id 465bcf0aa5fb22e5838145697eec67b9ae747d5a439d6ef77e300d4d560e5fea Sep 09 21:29:15.885554 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:15.885466 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88ddf69cd-w4qmv" event={"ID":"badb67c5-7566-4353-8736-b79a3732266f","Type":"ContainerStarted","Data":"465bcf0aa5fb22e5838145697eec67b9ae747d5a439d6ef77e300d4d560e5fea"} Sep 09 21:29:15.887283 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:15.887220 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-587659f668-k4jn6" event={"ID":"a770cafc-9c95-4517-9792-3796286cdfad","Type":"ContainerStarted","Data":"186431a02a51f8b74464c8668fe43fda64c349f7db1287ac411faa0f26a2aa02"} Sep 09 21:29:18.905805 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:18.905761 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-88ddf69cd-w4qmv" event={"ID":"badb67c5-7566-4353-8736-b79a3732266f","Type":"ContainerStarted","Data":"0dcc2ff898a030c3d5ee3d05c48aa51bec19a18cb98ce84a7185908ed1e74b69"} Sep 09 21:29:18.906320 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:18.905941 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:18.907497 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:18.907474 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-587659f668-k4jn6" event={"ID":"a770cafc-9c95-4517-9792-3796286cdfad","Type":"ContainerStarted","Data":"9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314"} Sep 09 21:29:18.907614 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:18.907523 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:18.928026 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:18.927974 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-88ddf69cd-w4qmv" podStartSLOduration=2.495681243 podStartE2EDuration="4.927958927s" totalImagesPullingTime="2.432277684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:29:18.926997307 +0000 UTC m=+1169.033741134" watchObservedRunningTime="2026-09-09 21:29:18.927958927 +0000 UTC m=+1169.034702752" Sep 09 21:29:18.947054 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:18.946999 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-587659f668-k4jn6" podStartSLOduration=2.339331401 podStartE2EDuration="4.946981832s" totalImagesPullingTime="2.607650431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:29:18.945003685 +0000 UTC m=+1169.051747536" watchObservedRunningTime="2026-09-09 21:29:18.946981832 +0000 UTC m=+1169.053725657" Sep 09 21:29:29.915991 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:29.915950 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-88ddf69cd-w4qmv" Sep 09 21:29:29.916428 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:29.916141 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:29.987905 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:29.987867 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-587659f668-k4jn6"] Sep 09 21:29:29.988223 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:29.988110 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-587659f668-k4jn6" podUID="a770cafc-9c95-4517-9792-3796286cdfad" containerName="manager" containerID="cri-o://9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314" gracePeriod=30 Sep 09 21:29:30.306927 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.306893 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:30.389645 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.389616 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vw52r\" (UniqueName: \"kubernetes.io/projected/a770cafc-9c95-4517-9792-3796286cdfad-kube-api-access-vw52r\") pod \"a770cafc-9c95-4517-9792-3796286cdfad\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " Sep 09 21:29:30.389850 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.389760 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-webhook-cert\") pod \"a770cafc-9c95-4517-9792-3796286cdfad\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " Sep 09 21:29:30.389850 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.389794 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-metrics-certs\") pod \"a770cafc-9c95-4517-9792-3796286cdfad\" (UID: \"a770cafc-9c95-4517-9792-3796286cdfad\") " Sep 09 21:29:30.391885 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.391856 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a770cafc-9c95-4517-9792-3796286cdfad" (UID: "a770cafc-9c95-4517-9792-3796286cdfad"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:29:30.391885 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.391876 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a770cafc-9c95-4517-9792-3796286cdfad" (UID: "a770cafc-9c95-4517-9792-3796286cdfad"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Sep 09 21:29:30.392049 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.391976 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a770cafc-9c95-4517-9792-3796286cdfad-kube-api-access-vw52r" (OuterVolumeSpecName: "kube-api-access-vw52r") pod "a770cafc-9c95-4517-9792-3796286cdfad" (UID: "a770cafc-9c95-4517-9792-3796286cdfad"). InnerVolumeSpecName "kube-api-access-vw52r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Sep 09 21:29:30.490737 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.490631 2553 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-webhook-cert\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:29:30.490737 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.490667 2553 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a770cafc-9c95-4517-9792-3796286cdfad-metrics-certs\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:29:30.490737 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.490681 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw52r\" (UniqueName: \"kubernetes.io/projected/a770cafc-9c95-4517-9792-3796286cdfad-kube-api-access-vw52r\") on node \"ip-10-0-137-115.ec2.internal\" DevicePath \"\"" Sep 09 21:29:30.988065 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.988034 2553 generic.go:356] "Generic (PLEG): container finished" podID="a770cafc-9c95-4517-9792-3796286cdfad" containerID="9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314" exitCode=0 Sep 09 21:29:30.988559 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.988077 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-587659f668-k4jn6" Sep 09 21:29:30.988559 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.988131 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-587659f668-k4jn6" event={"ID":"a770cafc-9c95-4517-9792-3796286cdfad","Type":"ContainerDied","Data":"9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314"} Sep 09 21:29:30.988559 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.988193 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-587659f668-k4jn6" event={"ID":"a770cafc-9c95-4517-9792-3796286cdfad","Type":"ContainerDied","Data":"186431a02a51f8b74464c8668fe43fda64c349f7db1287ac411faa0f26a2aa02"} Sep 09 21:29:30.988559 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.988225 2553 scope.go:117] "RemoveContainer" containerID="9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314" Sep 09 21:29:30.998079 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.998059 2553 scope.go:117] "RemoveContainer" containerID="9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314" Sep 09 21:29:30.998382 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:29:30.998363 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314\": container with ID starting with 9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314 not found: ID does not exist" containerID="9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314" Sep 09 21:29:30.998426 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:30.998391 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314"} err="failed to get container status \"9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314\": rpc error: code = NotFound desc = could not find container \"9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314\": container with ID starting with 9d1f8312f3911179eadda90d9c07cd40948d2ef71ce3044f8f97d3165f03b314 not found: ID does not exist" Sep 09 21:29:31.011328 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:31.011293 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-587659f668-k4jn6"] Sep 09 21:29:31.016635 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:31.016608 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-587659f668-k4jn6"] Sep 09 21:29:32.389998 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:32.389951 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a770cafc-9c95-4517-9792-3796286cdfad" path="/var/lib/kubelet/pods/a770cafc-9c95-4517-9792-3796286cdfad/volumes" Sep 09 21:29:50.424617 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:50.424589 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:29:50.426878 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:50.426854 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:29:52.312559 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.312517 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck"] Sep 09 21:29:52.313294 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.313204 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a770cafc-9c95-4517-9792-3796286cdfad" containerName="manager" Sep 09 21:29:52.313294 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.313262 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="a770cafc-9c95-4517-9792-3796286cdfad" containerName="manager" Sep 09 21:29:52.313574 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.313421 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="a770cafc-9c95-4517-9792-3796286cdfad" containerName="manager" Sep 09 21:29:52.318321 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.318300 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.320971 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.320951 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Sep 09 21:29:52.321073 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.320976 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Sep 09 21:29:52.326945 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.326920 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck"] Sep 09 21:29:52.396488 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.396454 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dpks\" (UniqueName: \"kubernetes.io/projected/a632cd8f-f012-4624-b71a-9817457e035e-kube-api-access-9dpks\") pod \"maas-api-6dfb4d7b79-dngck\" (UID: \"a632cd8f-f012-4624-b71a-9817457e035e\") " pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.396488 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.396494 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a632cd8f-f012-4624-b71a-9817457e035e-metrics-certs\") pod \"maas-api-6dfb4d7b79-dngck\" (UID: \"a632cd8f-f012-4624-b71a-9817457e035e\") " pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.396704 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.396519 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a632cd8f-f012-4624-b71a-9817457e035e-maas-api-tls\") pod \"maas-api-6dfb4d7b79-dngck\" (UID: \"a632cd8f-f012-4624-b71a-9817457e035e\") " pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.396704 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.396567 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a632cd8f-f012-4624-b71a-9817457e035e-openshift-service-ca\") pod \"maas-api-6dfb4d7b79-dngck\" (UID: \"a632cd8f-f012-4624-b71a-9817457e035e\") " pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.497175 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.497093 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a632cd8f-f012-4624-b71a-9817457e035e-openshift-service-ca\") pod \"maas-api-6dfb4d7b79-dngck\" (UID: \"a632cd8f-f012-4624-b71a-9817457e035e\") " pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.497340 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.497276 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dpks\" (UniqueName: \"kubernetes.io/projected/a632cd8f-f012-4624-b71a-9817457e035e-kube-api-access-9dpks\") pod \"maas-api-6dfb4d7b79-dngck\" (UID: \"a632cd8f-f012-4624-b71a-9817457e035e\") " pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.497340 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.497304 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a632cd8f-f012-4624-b71a-9817457e035e-metrics-certs\") pod \"maas-api-6dfb4d7b79-dngck\" (UID: \"a632cd8f-f012-4624-b71a-9817457e035e\") " pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.497456 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.497343 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a632cd8f-f012-4624-b71a-9817457e035e-maas-api-tls\") pod \"maas-api-6dfb4d7b79-dngck\" (UID: \"a632cd8f-f012-4624-b71a-9817457e035e\") " pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.497921 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.497898 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a632cd8f-f012-4624-b71a-9817457e035e-openshift-service-ca\") pod \"maas-api-6dfb4d7b79-dngck\" (UID: \"a632cd8f-f012-4624-b71a-9817457e035e\") " pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.499805 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.499783 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a632cd8f-f012-4624-b71a-9817457e035e-metrics-certs\") pod \"maas-api-6dfb4d7b79-dngck\" (UID: \"a632cd8f-f012-4624-b71a-9817457e035e\") " pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.499903 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.499814 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a632cd8f-f012-4624-b71a-9817457e035e-maas-api-tls\") pod \"maas-api-6dfb4d7b79-dngck\" (UID: \"a632cd8f-f012-4624-b71a-9817457e035e\") " pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.505648 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.505626 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dpks\" (UniqueName: \"kubernetes.io/projected/a632cd8f-f012-4624-b71a-9817457e035e-kube-api-access-9dpks\") pod \"maas-api-6dfb4d7b79-dngck\" (UID: \"a632cd8f-f012-4624-b71a-9817457e035e\") " pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.630124 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.630011 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:52.872393 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.872356 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4"] Sep 09 21:29:52.877133 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.877112 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" Sep 09 21:29:52.880310 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.880250 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-f98t7\"" Sep 09 21:29:52.880500 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.880487 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Sep 09 21:29:52.881214 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.881196 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-hplg2"] Sep 09 21:29:52.884773 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.884751 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" Sep 09 21:29:52.888921 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.888898 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4"] Sep 09 21:29:52.899276 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.899249 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-hplg2"] Sep 09 21:29:52.901426 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.901395 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xmsk\" (UniqueName: \"kubernetes.io/projected/034864b6-654b-458e-b3de-642b0302aa3d-kube-api-access-6xmsk\") pod \"payload-pre-processing-59fbc47bf8-kl8v4\" (UID: \"034864b6-654b-458e-b3de-642b0302aa3d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" Sep 09 21:29:52.901583 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.901502 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjh79\" (UniqueName: \"kubernetes.io/projected/df9b21aa-8185-40ce-8f17-dd23aed0e089-kube-api-access-mjh79\") pod \"payload-processing-5b54b7fbb7-hplg2\" (UID: \"df9b21aa-8185-40ce-8f17-dd23aed0e089\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" Sep 09 21:29:52.901583 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.901546 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df9b21aa-8185-40ce-8f17-dd23aed0e089-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-hplg2\" (UID: \"df9b21aa-8185-40ce-8f17-dd23aed0e089\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" Sep 09 21:29:52.901713 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.901648 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/034864b6-654b-458e-b3de-642b0302aa3d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-kl8v4\" (UID: \"034864b6-654b-458e-b3de-642b0302aa3d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" Sep 09 21:29:52.912974 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:52.912953 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck"] Sep 09 21:29:52.917218 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:29:52.917184 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda632cd8f_f012_4624_b71a_9817457e035e.slice/crio-cb1f1b43f9c999cf78d0006c7097bb29a826dfdd58e30616c8e3e321b45f12df WatchSource:0}: Error finding container cb1f1b43f9c999cf78d0006c7097bb29a826dfdd58e30616c8e3e321b45f12df: Status 404 returned error can't find the container with id cb1f1b43f9c999cf78d0006c7097bb29a826dfdd58e30616c8e3e321b45f12df Sep 09 21:29:53.002365 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.002330 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/034864b6-654b-458e-b3de-642b0302aa3d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-kl8v4\" (UID: \"034864b6-654b-458e-b3de-642b0302aa3d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" Sep 09 21:29:53.002584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.002394 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xmsk\" (UniqueName: \"kubernetes.io/projected/034864b6-654b-458e-b3de-642b0302aa3d-kube-api-access-6xmsk\") pod \"payload-pre-processing-59fbc47bf8-kl8v4\" (UID: \"034864b6-654b-458e-b3de-642b0302aa3d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" Sep 09 21:29:53.002584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.002427 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjh79\" (UniqueName: \"kubernetes.io/projected/df9b21aa-8185-40ce-8f17-dd23aed0e089-kube-api-access-mjh79\") pod \"payload-processing-5b54b7fbb7-hplg2\" (UID: \"df9b21aa-8185-40ce-8f17-dd23aed0e089\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" Sep 09 21:29:53.002584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.002451 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df9b21aa-8185-40ce-8f17-dd23aed0e089-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-hplg2\" (UID: \"df9b21aa-8185-40ce-8f17-dd23aed0e089\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" Sep 09 21:29:53.003211 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.003186 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df9b21aa-8185-40ce-8f17-dd23aed0e089-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-hplg2\" (UID: \"df9b21aa-8185-40ce-8f17-dd23aed0e089\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" Sep 09 21:29:53.003305 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.003194 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/034864b6-654b-458e-b3de-642b0302aa3d-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-kl8v4\" (UID: \"034864b6-654b-458e-b3de-642b0302aa3d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" Sep 09 21:29:53.020128 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.020072 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjh79\" (UniqueName: \"kubernetes.io/projected/df9b21aa-8185-40ce-8f17-dd23aed0e089-kube-api-access-mjh79\") pod \"payload-processing-5b54b7fbb7-hplg2\" (UID: \"df9b21aa-8185-40ce-8f17-dd23aed0e089\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" Sep 09 21:29:53.022712 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.022689 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xmsk\" (UniqueName: \"kubernetes.io/projected/034864b6-654b-458e-b3de-642b0302aa3d-kube-api-access-6xmsk\") pod \"payload-pre-processing-59fbc47bf8-kl8v4\" (UID: \"034864b6-654b-458e-b3de-642b0302aa3d\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" Sep 09 21:29:53.114084 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.114038 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" event={"ID":"a632cd8f-f012-4624-b71a-9817457e035e","Type":"ContainerStarted","Data":"cb1f1b43f9c999cf78d0006c7097bb29a826dfdd58e30616c8e3e321b45f12df"} Sep 09 21:29:53.192385 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.192354 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" Sep 09 21:29:53.201494 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.201470 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" Sep 09 21:29:53.472758 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.472732 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4"] Sep 09 21:29:53.474051 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:29:53.474022 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034864b6_654b_458e_b3de_642b0302aa3d.slice/crio-78f55efa685f7980f7df3066944f0d69f64776ff75e2ce20b075e6ce51b45320 WatchSource:0}: Error finding container 78f55efa685f7980f7df3066944f0d69f64776ff75e2ce20b075e6ce51b45320: Status 404 returned error can't find the container with id 78f55efa685f7980f7df3066944f0d69f64776ff75e2ce20b075e6ce51b45320 Sep 09 21:29:53.490875 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:53.490853 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-hplg2"] Sep 09 21:29:53.493219 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:29:53.493194 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf9b21aa_8185_40ce_8f17_dd23aed0e089.slice/crio-2a86efceed1b3e509e77e63b2fd3871a5d8934506b651a5129d6d6022212e78b WatchSource:0}: Error finding container 2a86efceed1b3e509e77e63b2fd3871a5d8934506b651a5129d6d6022212e78b: Status 404 returned error can't find the container with id 2a86efceed1b3e509e77e63b2fd3871a5d8934506b651a5129d6d6022212e78b Sep 09 21:29:54.125289 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:54.125247 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" event={"ID":"df9b21aa-8185-40ce-8f17-dd23aed0e089","Type":"ContainerStarted","Data":"2a86efceed1b3e509e77e63b2fd3871a5d8934506b651a5129d6d6022212e78b"} Sep 09 21:29:54.128008 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:54.127976 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" event={"ID":"034864b6-654b-458e-b3de-642b0302aa3d","Type":"ContainerStarted","Data":"78f55efa685f7980f7df3066944f0d69f64776ff75e2ce20b075e6ce51b45320"} Sep 09 21:29:57.183436 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:57.183405 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" event={"ID":"a632cd8f-f012-4624-b71a-9817457e035e","Type":"ContainerStarted","Data":"fe4cc3cd1d3e9990bf7d7b339c947ae5f39ccdeba0596e47072fc299e611b373"} Sep 09 21:29:57.183902 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:57.183767 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:29:57.185970 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:57.185941 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" event={"ID":"df9b21aa-8185-40ce-8f17-dd23aed0e089","Type":"ContainerStarted","Data":"7ac010bd2a21beca4cf41ba95393a07d69a83aa14b7256ad732755aa4c59e018"} Sep 09 21:29:57.186089 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:57.186037 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" Sep 09 21:29:57.187565 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:57.187540 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" event={"ID":"034864b6-654b-458e-b3de-642b0302aa3d","Type":"ContainerStarted","Data":"1f97940e7a0b8c21c3c895ee45bc41ec0f1498106f382fcfd3bc1370ab06b431"} Sep 09 21:29:57.187723 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:57.187660 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" Sep 09 21:29:57.202486 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:57.202435 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" podStartSLOduration=1.224651938 podStartE2EDuration="5.20242015s" totalImagesPullingTime="3.977768212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:29:57.200905786 +0000 UTC m=+1207.307649631" watchObservedRunningTime="2026-09-09 21:29:57.20242015 +0000 UTC m=+1207.309163975" Sep 09 21:29:57.216561 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:29:57.216385 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" podStartSLOduration=1.715184549 podStartE2EDuration="5.216366782s" totalImagesPullingTime="3.501182233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:29:57.216214826 +0000 UTC m=+1207.322958679" watchObservedRunningTime="2026-09-09 21:29:57.216366782 +0000 UTC m=+1207.323110616" Sep 09 21:30:03.198954 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:03.198916 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6dfb4d7b79-dngck" Sep 09 21:30:03.221203 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:03.221148 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" podStartSLOduration=7.746417563 podStartE2EDuration="11.221083433s" totalImagesPullingTime="3.47466587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:29:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:29:57.235228499 +0000 UTC m=+1207.341972343" watchObservedRunningTime="2026-09-09 21:30:03.221083433 +0000 UTC m=+1213.327827261" Sep 09 21:30:03.222691 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:03.222667 2553 generic.go:356] "Generic (PLEG): container finished" podID="3a4850ff-3bce-4850-abd4-17ae634f1fa3" containerID="95f1306c366e7e4e037746af7aa88a32261e9954df903d77dd643f1739f9b881" exitCode=0 Sep 09 21:30:03.222822 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:03.222743 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" event={"ID":"3a4850ff-3bce-4850-abd4-17ae634f1fa3","Type":"ContainerDied","Data":"95f1306c366e7e4e037746af7aa88a32261e9954df903d77dd643f1739f9b881"} Sep 09 21:30:03.223166 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:03.223150 2553 scope.go:117] "RemoveContainer" containerID="95f1306c366e7e4e037746af7aa88a32261e9954df903d77dd643f1739f9b881" Sep 09 21:30:03.224188 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:03.224154 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 21:30:03.224268 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:03.224221 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 21:30:03.224268 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:03.224254 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"} Sep 09 21:30:04.230046 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:04.230012 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" event={"ID":"3a4850ff-3bce-4850-abd4-17ae634f1fa3","Type":"ContainerStarted","Data":"d4cfddbba5e7ad272c9ce5ca89602f8ad18db5373268860e6609863e92262a76"} Sep 09 21:30:05.166276 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:05.166238 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:30:05.167033 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:05.167004 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hplk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 09 21:30:05.167176 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:05.167051 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" podUID="3a4850ff-3bce-4850-abd4-17ae634f1fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 09 21:30:06.166531 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:06.166495 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hplk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 09 21:30:06.167138 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:06.167071 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" podUID="3a4850ff-3bce-4850-abd4-17ae634f1fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 09 21:30:07.165888 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:07.165847 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hplk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 09 21:30:07.166211 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:07.165902 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" podUID="3a4850ff-3bce-4850-abd4-17ae634f1fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 09 21:30:07.592691 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:07.592646 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:30:08.165976 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:08.165936 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hplk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 09 21:30:08.166209 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:08.166007 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" podUID="3a4850ff-3bce-4850-abd4-17ae634f1fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 09 21:30:08.196433 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:08.196095 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-kl8v4" Sep 09 21:30:08.196433 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:08.196371 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-hplg2" Sep 09 21:30:09.165634 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:09.165592 2553 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hplk6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Sep 09 21:30:09.166032 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:09.165662 2553 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" podUID="3a4850ff-3bce-4850-abd4-17ae634f1fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Sep 09 21:30:10.170442 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:10.170408 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:30:10.171637 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:10.171615 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hplk6" Sep 09 21:30:16.590238 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.590205 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf"] Sep 09 21:30:16.595571 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.595547 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.598316 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.598289 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Sep 09 21:30:16.598448 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.598334 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Sep 09 21:30:16.598448 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.598389 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6rwsm\"" Sep 09 21:30:16.598568 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.598483 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Sep 09 21:30:16.604031 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.603717 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf"] Sep 09 21:30:16.676126 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.676083 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.676126 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.676132 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.676402 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.676158 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mscr\" (UniqueName: \"kubernetes.io/projected/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-kube-api-access-5mscr\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.676402 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.676189 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.676402 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.676211 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.676402 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.676303 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.777376 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.777342 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.777376 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.777384 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.777622 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.777400 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mscr\" (UniqueName: \"kubernetes.io/projected/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-kube-api-access-5mscr\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.777622 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.777433 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.777622 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.777467 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.777622 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.777540 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.777852 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.777826 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-home\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.777852 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.777844 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-tmp-dir\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.777964 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.777882 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-model-cache\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.779788 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.779751 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-dshm\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.780030 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.780013 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-tls-certs\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.785782 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.785758 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mscr\" (UniqueName: \"kubernetes.io/projected/02188189-a3cc-4ef7-812a-5edb6ec3e3c2-kube-api-access-5mscr\") pod \"e2e-distinct-simulated-kserve-6685d86975-zgwnf\" (UID: \"02188189-a3cc-4ef7-812a-5edb6ec3e3c2\") " pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:16.909360 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:16.909262 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:17.186875 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:17.186658 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf"] Sep 09 21:30:17.189048 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:30:17.189004 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02188189_a3cc_4ef7_812a_5edb6ec3e3c2.slice/crio-f252c1c4ca544571c8d1301fd1b96ebd3519a621d3cdbcd1a3d06d45bcfa8947 WatchSource:0}: Error finding container f252c1c4ca544571c8d1301fd1b96ebd3519a621d3cdbcd1a3d06d45bcfa8947: Status 404 returned error can't find the container with id f252c1c4ca544571c8d1301fd1b96ebd3519a621d3cdbcd1a3d06d45bcfa8947 Sep 09 21:30:17.325861 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:17.325830 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" event={"ID":"02188189-a3cc-4ef7-812a-5edb6ec3e3c2","Type":"ContainerStarted","Data":"f252c1c4ca544571c8d1301fd1b96ebd3519a621d3cdbcd1a3d06d45bcfa8947"} Sep 09 21:30:18.334772 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:18.334732 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" event={"ID":"02188189-a3cc-4ef7-812a-5edb6ec3e3c2","Type":"ContainerStarted","Data":"279b8fb2f3ee7eb736be99b3bc07812af57ee8e421822b5c4ff524c5cc70ac84"} Sep 09 21:30:18.353541 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:18.353486 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podStartSLOduration=1.308810509 podStartE2EDuration="2.353470492s" totalImagesPullingTime="1.044659983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:30:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:30:18.352867989 +0000 UTC m=+1228.459611848" watchObservedRunningTime="2026-09-09 21:30:18.353470492 +0000 UTC m=+1228.460214317" Sep 09 21:30:19.344005 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:19.343973 2553 generic.go:356] "Generic (PLEG): container finished" podID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" containerID="279b8fb2f3ee7eb736be99b3bc07812af57ee8e421822b5c4ff524c5cc70ac84" exitCode=0 Sep 09 21:30:19.344443 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:19.344067 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" event={"ID":"02188189-a3cc-4ef7-812a-5edb6ec3e3c2","Type":"ContainerDied","Data":"279b8fb2f3ee7eb736be99b3bc07812af57ee8e421822b5c4ff524c5cc70ac84"} Sep 09 21:30:19.344510 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:19.344492 2553 scope.go:117] "RemoveContainer" containerID="279b8fb2f3ee7eb736be99b3bc07812af57ee8e421822b5c4ff524c5cc70ac84" Sep 09 21:30:20.352216 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:20.352175 2553 generic.go:356] "Generic (PLEG): container finished" podID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" containerID="d9becd1eb7cb101b845fcda585303e638a5a1e3ddfea06c97451587b727e9ea8" exitCode=0 Sep 09 21:30:20.352687 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:20.352251 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" event={"ID":"02188189-a3cc-4ef7-812a-5edb6ec3e3c2","Type":"ContainerDied","Data":"d9becd1eb7cb101b845fcda585303e638a5a1e3ddfea06c97451587b727e9ea8"} Sep 09 21:30:20.352687 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:20.352314 2553 scope.go:117] "RemoveContainer" containerID="279b8fb2f3ee7eb736be99b3bc07812af57ee8e421822b5c4ff524c5cc70ac84" Sep 09 21:30:20.352687 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:20.352613 2553 scope.go:117] "RemoveContainer" containerID="d9becd1eb7cb101b845fcda585303e638a5a1e3ddfea06c97451587b727e9ea8" Sep 09 21:30:20.352848 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:30:20.352829 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:30:21.359896 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:21.359867 2553 scope.go:117] "RemoveContainer" containerID="d9becd1eb7cb101b845fcda585303e638a5a1e3ddfea06c97451587b727e9ea8" Sep 09 21:30:21.360316 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:30:21.360049 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:30:26.909646 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:26.909598 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:26.910056 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:26.909692 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:26.910056 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:26.909710 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:26.910056 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:26.909722 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:26.910201 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:26.910174 2553 scope.go:117] "RemoveContainer" containerID="d9becd1eb7cb101b845fcda585303e638a5a1e3ddfea06c97451587b727e9ea8" Sep 09 21:30:26.910391 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:30:26.910371 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 10s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:30:37.303721 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.303684 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2"] Sep 09 21:30:37.310293 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.310256 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.313031 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.313003 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Sep 09 21:30:37.320348 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.320325 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2"] Sep 09 21:30:37.384445 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.384412 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcmcx\" (UniqueName: \"kubernetes.io/projected/296db7f9-ed6a-48ba-a8d9-69781c7ea930-kube-api-access-wcmcx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.384445 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.384451 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/296db7f9-ed6a-48ba-a8d9-69781c7ea930-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.384718 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.384478 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/296db7f9-ed6a-48ba-a8d9-69781c7ea930-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.384718 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.384496 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/296db7f9-ed6a-48ba-a8d9-69781c7ea930-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.384718 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.384604 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/296db7f9-ed6a-48ba-a8d9-69781c7ea930-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.384718 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.384680 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/296db7f9-ed6a-48ba-a8d9-69781c7ea930-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.485985 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.485954 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/296db7f9-ed6a-48ba-a8d9-69781c7ea930-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.486231 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.486002 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/296db7f9-ed6a-48ba-a8d9-69781c7ea930-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.486231 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.486060 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/296db7f9-ed6a-48ba-a8d9-69781c7ea930-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.486231 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.486194 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/296db7f9-ed6a-48ba-a8d9-69781c7ea930-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.486487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.486280 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/296db7f9-ed6a-48ba-a8d9-69781c7ea930-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.486487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.486365 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcmcx\" (UniqueName: \"kubernetes.io/projected/296db7f9-ed6a-48ba-a8d9-69781c7ea930-kube-api-access-wcmcx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.486487 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.486422 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/296db7f9-ed6a-48ba-a8d9-69781c7ea930-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.486652 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.486552 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/296db7f9-ed6a-48ba-a8d9-69781c7ea930-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.486715 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.486701 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/296db7f9-ed6a-48ba-a8d9-69781c7ea930-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.488741 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.488711 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/296db7f9-ed6a-48ba-a8d9-69781c7ea930-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.488936 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.488922 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/296db7f9-ed6a-48ba-a8d9-69781c7ea930-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.500582 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.500544 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcmcx\" (UniqueName: \"kubernetes.io/projected/296db7f9-ed6a-48ba-a8d9-69781c7ea930-kube-api-access-wcmcx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-b2nh2\" (UID: \"296db7f9-ed6a-48ba-a8d9-69781c7ea930\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.622605 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.622509 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:37.898274 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.898244 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2"] Sep 09 21:30:37.900380 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:30:37.900346 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod296db7f9_ed6a_48ba_a8d9_69781c7ea930.slice/crio-8fc309f607f2bcdbf072c7df508660efb023106487a374f9966a593c2ce303ee WatchSource:0}: Error finding container 8fc309f607f2bcdbf072c7df508660efb023106487a374f9966a593c2ce303ee: Status 404 returned error can't find the container with id 8fc309f607f2bcdbf072c7df508660efb023106487a374f9966a593c2ce303ee Sep 09 21:30:37.901832 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:37.901813 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Sep 09 21:30:38.456160 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:38.456121 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" event={"ID":"296db7f9-ed6a-48ba-a8d9-69781c7ea930","Type":"ContainerStarted","Data":"4aace2a70bc9b2dcf76e3cfa58f95fd4ca9846ac7c1a4a537bf7f19c828e6870"} Sep 09 21:30:38.456582 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:38.456174 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" event={"ID":"296db7f9-ed6a-48ba-a8d9-69781c7ea930","Type":"ContainerStarted","Data":"8fc309f607f2bcdbf072c7df508660efb023106487a374f9966a593c2ce303ee"} Sep 09 21:30:38.475520 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:38.475467 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" podStartSLOduration=1.271936711 podStartE2EDuration="1.475451343s" totalImagesPullingTime="203.514632ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 21:30:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:30:38.474488885 +0000 UTC m=+1248.581232764" watchObservedRunningTime="2026-09-09 21:30:38.475451343 +0000 UTC m=+1248.582195193" Sep 09 21:30:41.384136 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:41.384085 2553 scope.go:117] "RemoveContainer" containerID="d9becd1eb7cb101b845fcda585303e638a5a1e3ddfea06c97451587b727e9ea8" Sep 09 21:30:42.487352 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:42.487274 2553 generic.go:356] "Generic (PLEG): container finished" podID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" containerID="dd8003ad6014febcd5a6513f1d47efc04a9f4465519f3e3500e625015ac816cf" exitCode=0 Sep 09 21:30:42.487352 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:42.487305 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" event={"ID":"02188189-a3cc-4ef7-812a-5edb6ec3e3c2","Type":"ContainerDied","Data":"dd8003ad6014febcd5a6513f1d47efc04a9f4465519f3e3500e625015ac816cf"} Sep 09 21:30:42.487854 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:42.487393 2553 scope.go:117] "RemoveContainer" containerID="d9becd1eb7cb101b845fcda585303e638a5a1e3ddfea06c97451587b727e9ea8" Sep 09 21:30:42.487854 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:42.487799 2553 scope.go:117] "RemoveContainer" containerID="dd8003ad6014febcd5a6513f1d47efc04a9f4465519f3e3500e625015ac816cf" Sep 09 21:30:42.488034 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:30:42.488011 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:30:46.911999 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:46.910431 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:46.911999 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:46.910522 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:46.911999 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:46.911086 2553 scope.go:117] "RemoveContainer" containerID="dd8003ad6014febcd5a6513f1d47efc04a9f4465519f3e3500e625015ac816cf" Sep 09 21:30:46.911999 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:30:46.911363 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:30:46.912958 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:46.912938 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:46.913131 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:46.913115 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:30:47.524584 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:47.524549 2553 scope.go:117] "RemoveContainer" containerID="dd8003ad6014febcd5a6513f1d47efc04a9f4465519f3e3500e625015ac816cf" Sep 09 21:30:47.525279 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:30:47.524842 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:30:47.623231 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:47.623195 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:47.623457 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:47.623256 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:47.640520 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:47.640484 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:30:48.529935 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:48.529907 2553 scope.go:117] "RemoveContainer" containerID="dd8003ad6014febcd5a6513f1d47efc04a9f4465519f3e3500e625015ac816cf" Sep 09 21:30:48.530372 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:30:48.530138 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:30:48.541404 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:30:48.541374 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-b2nh2" Sep 09 21:31:01.384310 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:01.384275 2553 scope.go:117] "RemoveContainer" containerID="dd8003ad6014febcd5a6513f1d47efc04a9f4465519f3e3500e625015ac816cf" Sep 09 21:31:01.384739 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:31:01.384470 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:31:12.384407 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:12.384369 2553 scope.go:117] "RemoveContainer" containerID="dd8003ad6014febcd5a6513f1d47efc04a9f4465519f3e3500e625015ac816cf" Sep 09 21:31:13.673975 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:13.673939 2553 generic.go:356] "Generic (PLEG): container finished" podID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" containerID="2de2a30eba464e7054d2d7af9d36026979c9a31bf1018a616cc46b3b6831532d" exitCode=0 Sep 09 21:31:13.674465 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:13.673996 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" event={"ID":"02188189-a3cc-4ef7-812a-5edb6ec3e3c2","Type":"ContainerDied","Data":"2de2a30eba464e7054d2d7af9d36026979c9a31bf1018a616cc46b3b6831532d"} Sep 09 21:31:13.674465 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:13.674046 2553 scope.go:117] "RemoveContainer" containerID="dd8003ad6014febcd5a6513f1d47efc04a9f4465519f3e3500e625015ac816cf" Sep 09 21:31:13.674597 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:13.674501 2553 scope.go:117] "RemoveContainer" containerID="2de2a30eba464e7054d2d7af9d36026979c9a31bf1018a616cc46b3b6831532d" Sep 09 21:31:13.674783 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:31:13.674761 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:31:16.909556 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:16.909507 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:31:16.910005 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:16.909583 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:31:16.910005 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:16.909607 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:31:16.910005 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:16.909624 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:31:16.910005 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:16.909997 2553 scope.go:117] "RemoveContainer" containerID="2de2a30eba464e7054d2d7af9d36026979c9a31bf1018a616cc46b3b6831532d" Sep 09 21:31:16.910239 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:31:16.910213 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:31:30.386795 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:30.386760 2553 scope.go:117] "RemoveContainer" containerID="2de2a30eba464e7054d2d7af9d36026979c9a31bf1018a616cc46b3b6831532d" Sep 09 21:31:30.387339 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:31:30.387013 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:31:41.384270 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:41.384232 2553 scope.go:117] "RemoveContainer" containerID="2de2a30eba464e7054d2d7af9d36026979c9a31bf1018a616cc46b3b6831532d" Sep 09 21:31:41.384679 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:31:41.384462 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:31:56.383786 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:56.383750 2553 scope.go:117] "RemoveContainer" containerID="2de2a30eba464e7054d2d7af9d36026979c9a31bf1018a616cc46b3b6831532d" Sep 09 21:31:56.914909 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:56.914817 2553 generic.go:356] "Generic (PLEG): container finished" podID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" containerID="7ba63de84d483987eb6378d78b0e5121ea505beba0c9962cac59461424fb1dea" exitCode=0 Sep 09 21:31:56.914909 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:56.914850 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" event={"ID":"02188189-a3cc-4ef7-812a-5edb6ec3e3c2","Type":"ContainerDied","Data":"7ba63de84d483987eb6378d78b0e5121ea505beba0c9962cac59461424fb1dea"} Sep 09 21:31:56.915134 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:56.914937 2553 scope.go:117] "RemoveContainer" containerID="2de2a30eba464e7054d2d7af9d36026979c9a31bf1018a616cc46b3b6831532d" Sep 09 21:31:56.915327 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:31:56.915312 2553 scope.go:117] "RemoveContainer" containerID="7ba63de84d483987eb6378d78b0e5121ea505beba0c9962cac59461424fb1dea" Sep 09 21:31:56.915574 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:31:56.915551 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:32:06.909730 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:32:06.909690 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:32:06.910224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:32:06.909767 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:32:06.910224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:32:06.909785 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:32:06.910224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:32:06.909804 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:32:06.910224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:32:06.910202 2553 scope.go:117] "RemoveContainer" containerID="7ba63de84d483987eb6378d78b0e5121ea505beba0c9962cac59461424fb1dea" Sep 09 21:32:06.910409 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:32:06.910390 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:32:06.971092 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:32:06.971063 2553 scope.go:117] "RemoveContainer" containerID="7ba63de84d483987eb6378d78b0e5121ea505beba0c9962cac59461424fb1dea" Sep 09 21:32:06.971337 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:32:06.971314 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:32:18.383980 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:32:18.383898 2553 scope.go:117] "RemoveContainer" containerID="7ba63de84d483987eb6378d78b0e5121ea505beba0c9962cac59461424fb1dea" Sep 09 21:32:18.384398 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:32:18.384168 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:32:31.383721 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:32:31.383681 2553 scope.go:117] "RemoveContainer" containerID="7ba63de84d483987eb6378d78b0e5121ea505beba0c9962cac59461424fb1dea" Sep 09 21:32:31.384184 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:32:31.383858 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:32:44.384392 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:32:44.384359 2553 scope.go:117] "RemoveContainer" containerID="7ba63de84d483987eb6378d78b0e5121ea505beba0c9962cac59461424fb1dea" Sep 09 21:32:44.384877 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:32:44.384559 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:32:58.383688 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:32:58.383650 2553 scope.go:117] "RemoveContainer" containerID="7ba63de84d483987eb6378d78b0e5121ea505beba0c9962cac59461424fb1dea" Sep 09 21:32:58.384196 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:32:58.383884 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:33:12.384265 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:12.384230 2553 scope.go:117] "RemoveContainer" containerID="7ba63de84d483987eb6378d78b0e5121ea505beba0c9962cac59461424fb1dea" Sep 09 21:33:12.384683 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:33:12.384430 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:33:25.383473 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:25.383440 2553 scope.go:117] "RemoveContainer" containerID="7ba63de84d483987eb6378d78b0e5121ea505beba0c9962cac59461424fb1dea" Sep 09 21:33:26.413007 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:26.412968 2553 generic.go:356] "Generic (PLEG): container finished" podID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" exitCode=0 Sep 09 21:33:26.413539 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:26.413041 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" event={"ID":"02188189-a3cc-4ef7-812a-5edb6ec3e3c2","Type":"ContainerDied","Data":"6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f"} Sep 09 21:33:26.413539 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:26.413110 2553 scope.go:117] "RemoveContainer" containerID="7ba63de84d483987eb6378d78b0e5121ea505beba0c9962cac59461424fb1dea" Sep 09 21:33:26.413666 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:26.413573 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:33:26.413810 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:33:26.413788 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:33:26.910365 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:26.910326 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:33:26.910558 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:26.910400 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:33:26.910558 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:26.910427 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:33:26.910558 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:26.910441 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" Sep 09 21:33:27.420520 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:27.420492 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:33:27.420928 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:33:27.420690 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:33:43.384543 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:43.384462 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:33:43.384981 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:33:43.384622 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:33:58.384203 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:33:58.384169 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:33:58.384689 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:33:58.384371 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:34:13.384074 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:34:13.384041 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:34:13.384501 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:34:13.384274 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:34:25.383589 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:34:25.383553 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:34:25.384028 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:34:25.383739 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:34:37.383321 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:34:37.383286 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:34:37.383752 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:34:37.383464 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:34:50.469203 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:34:50.469175 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:34:50.471157 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:34:50.471131 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:34:52.384419 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:34:52.384390 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:34:52.384879 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:34:52.384565 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:35:04.383711 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:04.383673 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:35:04.386244 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:35:04.383924 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:35:14.726694 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:14.726606 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lqck6_43cda0ce-a5ed-411b-b506-76e212d3bafc/manager/0.log" Sep 09 21:35:14.839323 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:14.839294 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vj7dn_4537c0f1-9d46-4c75-bcb2-1996f5497127/manager/0.log" Sep 09 21:35:14.957569 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:14.957536 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-88ddf69cd-w4qmv_badb67c5-7566-4353-8736-b79a3732266f/manager/0.log" Sep 09 21:35:15.464284 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:15.464254 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66dbc47bfc-nqxgt_a66d6957-d93d-4562-bb3a-e2504df76a07/copy-manifests/0.log" Sep 09 21:35:15.473905 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:15.473863 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66dbc47bfc-nqxgt_a66d6957-d93d-4562-bb3a-e2504df76a07/manager/0.log" Sep 09 21:35:15.961953 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:15.961919 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6dfb4d7b79-dngck_a632cd8f-f012-4624-b71a-9817457e035e/maas-api/0.log" Sep 09 21:35:16.189022 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:16.188993 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-5x52n_331b37ea-9053-4a2b-b1ff-973775eacc06/postgres/0.log" Sep 09 21:35:16.384480 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:16.384394 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:35:16.384638 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:35:16.384581 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:35:17.080999 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:17.080966 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-svmkk_6a030bb6-6577-47e1-a370-aee5ccada074/manager/0.log" Sep 09 21:35:17.201148 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:17.201113 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-bkqqq_be3faeaa-0256-4c16-9c07-76ead90b33ae/manager/0.log" Sep 09 21:35:17.427274 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:17.427237 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-7mwx2_db31f506-8059-4d16-bb7e-98adb21a9350/manager/0.log" Sep 09 21:35:17.676127 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:17.676083 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-wgfgp_23aad71c-05cb-4a06-a24c-764c9ea1cb1e/manager/0.log" Sep 09 21:35:18.255723 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:18.255674 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-696d6796dd-thz62_51a8f793-9fae-4a02-bfc3-c5e42c7aab94/kube-auth-proxy/0.log" Sep 09 21:35:18.486632 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:18.486587 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hplk6_3a4850ff-3bce-4850-abd4-17ae634f1fa3/istio-proxy/1.log" Sep 09 21:35:18.597128 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:18.597007 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-kl8v4_034864b6-654b-458e-b3de-642b0302aa3d/payload-pre-processing/0.log" Sep 09 21:35:18.708739 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:18.708672 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hplg2_df9b21aa-8185-40ce-8f17-dd23aed0e089/payload-processing/0.log" Sep 09 21:35:18.833483 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:18.833450 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64f7976768-h7vtq_05b70482-3a69-427f-8ce3-922c764c470b/router/0.log" Sep 09 21:35:19.307923 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:19.307894 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-6685d86975-zgwnf_02188189-a3cc-4ef7-812a-5edb6ec3e3c2/main/5.log" Sep 09 21:35:19.545001 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:19.544971 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-b2nh2_296db7f9-ed6a-48ba-a8d9-69781c7ea930/main/0.log" Sep 09 21:35:28.710305 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:28.710273 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tbgn7_a77a1afc-9eda-42b4-857c-2967b6d149d8/global-pull-secret-syncer/0.log" Sep 09 21:35:28.823820 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:28.823773 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bf4gk_f4e4311b-1130-4e07-8052-599cbb8ef158/konnectivity-agent/0.log" Sep 09 21:35:28.915928 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:28.915903 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-115.ec2.internal_0531880037de699c667aef422549784e/haproxy/0.log" Sep 09 21:35:30.386284 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:30.386254 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:35:30.386681 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:35:30.386442 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:35:33.552927 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:33.552896 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-svmkk_6a030bb6-6577-47e1-a370-aee5ccada074/manager/0.log" Sep 09 21:35:33.582542 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:33.582494 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-bkqqq_be3faeaa-0256-4c16-9c07-76ead90b33ae/manager/0.log" Sep 09 21:35:33.674945 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:33.674897 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-7mwx2_db31f506-8059-4d16-bb7e-98adb21a9350/manager/0.log" Sep 09 21:35:33.734837 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:33.734801 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-wgfgp_23aad71c-05cb-4a06-a24c-764c9ea1cb1e/manager/0.log" Sep 09 21:35:35.343075 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:35.343048 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-6pg2n_4cfb52c0-2216-418e-adf2-80859d555aa9/cluster-monitoring-operator/0.log" Sep 09 21:35:35.371344 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:35.371309 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-fwmw6_26d8967d-d973-40cb-b7a8-fa88046c521a/kube-state-metrics/0.log" Sep 09 21:35:35.397005 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:35.396978 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-fwmw6_26d8967d-d973-40cb-b7a8-fa88046c521a/kube-rbac-proxy-main/0.log" Sep 09 21:35:35.419235 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:35.419202 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-fwmw6_26d8967d-d973-40cb-b7a8-fa88046c521a/kube-rbac-proxy-self/0.log" Sep 09 21:35:35.496073 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:35.496044 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-pzl2v_ab209ce4-7847-40aa-886c-e4e5aaf19f1a/monitoring-plugin/0.log" Sep 09 21:35:35.723781 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:35.723757 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p9lg4_b1c6e518-b7ac-4e11-bc64-f67e8d0eddae/node-exporter/0.log" Sep 09 21:35:35.761050 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:35.761026 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p9lg4_b1c6e518-b7ac-4e11-bc64-f67e8d0eddae/kube-rbac-proxy/0.log" Sep 09 21:35:35.786684 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:35.786662 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-p9lg4_b1c6e518-b7ac-4e11-bc64-f67e8d0eddae/init-textfile/0.log" Sep 09 21:35:35.907201 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:35.907091 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_527baa92-bd10-4f5d-a15c-b68cde8e0642/prometheus/0.log" Sep 09 21:35:35.933536 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:35.933498 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_527baa92-bd10-4f5d-a15c-b68cde8e0642/config-reloader/0.log" Sep 09 21:35:35.960308 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:35.960283 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_527baa92-bd10-4f5d-a15c-b68cde8e0642/thanos-sidecar/0.log" Sep 09 21:35:35.987429 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:35.987398 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_527baa92-bd10-4f5d-a15c-b68cde8e0642/kube-rbac-proxy-web/0.log" Sep 09 21:35:36.018508 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.018483 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_527baa92-bd10-4f5d-a15c-b68cde8e0642/kube-rbac-proxy/0.log" Sep 09 21:35:36.068721 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.068690 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_527baa92-bd10-4f5d-a15c-b68cde8e0642/kube-rbac-proxy-thanos/0.log" Sep 09 21:35:36.126432 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.126390 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_527baa92-bd10-4f5d-a15c-b68cde8e0642/init-config-reloader/0.log" Sep 09 21:35:36.199705 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.199666 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-v6xj7_243e928c-cca9-496e-b035-496219854fa0/prometheus-operator/0.log" Sep 09 21:35:36.232558 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.232532 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-v6xj7_243e928c-cca9-496e-b035-496219854fa0/kube-rbac-proxy/0.log" Sep 09 21:35:36.265834 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.265802 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-78f599db79-wvt6t_91a28cbd-41bf-44d4-8503-2aefc141c3e3/prometheus-operator-admission-webhook/0.log" Sep 09 21:35:36.302811 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.302768 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84688c96dd-bwqdl_0ffa260b-9f12-44e6-823d-a4ed52e2bafa/telemeter-client/0.log" Sep 09 21:35:36.333037 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.333008 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84688c96dd-bwqdl_0ffa260b-9f12-44e6-823d-a4ed52e2bafa/reload/0.log" Sep 09 21:35:36.356479 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.356413 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84688c96dd-bwqdl_0ffa260b-9f12-44e6-823d-a4ed52e2bafa/kube-rbac-proxy/0.log" Sep 09 21:35:36.389701 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.389665 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ddd999fd7-tsjkv_be52978f-2862-4b78-ac25-e85d314114da/thanos-query/0.log" Sep 09 21:35:36.416409 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.416382 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ddd999fd7-tsjkv_be52978f-2862-4b78-ac25-e85d314114da/kube-rbac-proxy-web/0.log" Sep 09 21:35:36.439024 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.439002 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ddd999fd7-tsjkv_be52978f-2862-4b78-ac25-e85d314114da/kube-rbac-proxy/0.log" Sep 09 21:35:36.462014 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.461944 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ddd999fd7-tsjkv_be52978f-2862-4b78-ac25-e85d314114da/prom-label-proxy/0.log" Sep 09 21:35:36.489539 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.489511 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ddd999fd7-tsjkv_be52978f-2862-4b78-ac25-e85d314114da/kube-rbac-proxy-rules/0.log" Sep 09 21:35:36.511841 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:36.511787 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6ddd999fd7-tsjkv_be52978f-2862-4b78-ac25-e85d314114da/kube-rbac-proxy-metrics/0.log" Sep 09 21:35:37.755215 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:37.755186 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-9hd77_7bd30f8a-b08d-444d-b317-a32e7e0152b3/networking-console-plugin/0.log" Sep 09 21:35:38.347503 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:38.347472 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/2.log" Sep 09 21:35:38.358213 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:38.358184 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-9gxmg_6cec47f9-1808-4396-9ee3-4fc7c5cf1586/console-operator/3.log" Sep 09 21:35:38.961445 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:38.961412 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-cp7lf_fbfbe2bc-0541-4fc5-9b46-aee61a222398/download-server/0.log" Sep 09 21:35:39.663664 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:39.663632 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-r5sd7_097d572f-d83a-493b-a181-59385bb2afc1/volume-data-source-validator/0.log" Sep 09 21:35:39.898416 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:39.898383 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst"] Sep 09 21:35:39.902249 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:39.902231 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:39.904745 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:39.904718 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-znttn\"/\"default-dockercfg-96mk2\"" Sep 09 21:35:39.904968 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:39.904951 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-znttn\"/\"openshift-service-ca.crt\"" Sep 09 21:35:39.905720 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:39.905704 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-znttn\"/\"kube-root-ca.crt\"" Sep 09 21:35:39.913009 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:39.912979 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst"] Sep 09 21:35:39.973957 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:39.973926 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-proc\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:39.974392 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:39.973972 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-podres\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:39.974392 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:39.974034 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-sys\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:39.974392 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:39.974133 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-lib-modules\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:39.974392 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:39.974158 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tr6k\" (UniqueName: \"kubernetes.io/projected/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-kube-api-access-2tr6k\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:40.075442 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.075399 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-proc\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:40.075644 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.075462 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-podres\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:40.075644 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.075500 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-sys\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:40.075644 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.075527 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-proc\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:40.075644 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.075550 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-lib-modules\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:40.075644 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.075577 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-sys\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:40.075644 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.075579 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tr6k\" (UniqueName: \"kubernetes.io/projected/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-kube-api-access-2tr6k\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:40.075953 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.075639 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-podres\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:40.075953 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.075654 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-lib-modules\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:40.084637 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.084607 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tr6k\" (UniqueName: \"kubernetes.io/projected/1c6b9450-d43d-47de-9fe7-ea5249a16f1c-kube-api-access-2tr6k\") pod \"perf-node-gather-daemonset-dhdst\" (UID: \"1c6b9450-d43d-47de-9fe7-ea5249a16f1c\") " pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:40.213162 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.213132 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:40.487167 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.487138 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst"] Sep 09 21:35:40.488310 ip-10-0-137-115 kubenswrapper[2553]: W0909 21:35:40.488287 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1c6b9450_d43d_47de_9fe7_ea5249a16f1c.slice/crio-736796fa642afb61abb84d5c8c8f9c0c7bd55424ad99a4e17e9283f7020c22d6 WatchSource:0}: Error finding container 736796fa642afb61abb84d5c8c8f9c0c7bd55424ad99a4e17e9283f7020c22d6: Status 404 returned error can't find the container with id 736796fa642afb61abb84d5c8c8f9c0c7bd55424ad99a4e17e9283f7020c22d6 Sep 09 21:35:40.915572 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.915485 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qxszf_ca9ee75e-2df9-47f5-b556-977f41f7eeb1/dns/0.log" Sep 09 21:35:40.937033 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:40.937005 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qxszf_ca9ee75e-2df9-47f5-b556-977f41f7eeb1/kube-rbac-proxy/0.log" Sep 09 21:35:41.036772 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:41.036736 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sz8l8_91388845-f81d-42fa-b7ec-5e50730971ea/dns-node-resolver/0.log" Sep 09 21:35:41.153726 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:41.153683 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" event={"ID":"1c6b9450-d43d-47de-9fe7-ea5249a16f1c","Type":"ContainerStarted","Data":"dfc2940d009de79e1aab66bde1ca962df4d4d20882c703c2db9e73d9793000be"} Sep 09 21:35:41.153924 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:41.153760 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" event={"ID":"1c6b9450-d43d-47de-9fe7-ea5249a16f1c","Type":"ContainerStarted","Data":"736796fa642afb61abb84d5c8c8f9c0c7bd55424ad99a4e17e9283f7020c22d6"} Sep 09 21:35:41.153924 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:41.153794 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:41.193784 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:41.193736 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" podStartSLOduration=2.193718522 podStartE2EDuration="2.193718522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 21:35:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 21:35:41.19290339 +0000 UTC m=+1551.299647238" watchObservedRunningTime="2026-09-09 21:35:41.193718522 +0000 UTC m=+1551.300462369" Sep 09 21:35:41.384215 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:41.384180 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:35:41.384405 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:35:41.384384 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:35:41.687728 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:41.687704 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-srt4v_436fa819-8e30-43d8-ae10-7c9d0cbc04c5/node-ca/0.log" Sep 09 21:35:42.752605 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:42.752570 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-696d6796dd-thz62_51a8f793-9fae-4a02-bfc3-c5e42c7aab94/kube-auth-proxy/0.log" Sep 09 21:35:42.809051 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:42.809025 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hplk6_3a4850ff-3bce-4850-abd4-17ae634f1fa3/istio-proxy/1.log" Sep 09 21:35:42.809486 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:42.809467 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hplk6_3a4850ff-3bce-4850-abd4-17ae634f1fa3/istio-proxy/0.log" Sep 09 21:35:42.829748 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:42.829723 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-kl8v4_034864b6-654b-458e-b3de-642b0302aa3d/payload-pre-processing/0.log" Sep 09 21:35:42.854334 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:42.854305 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hplg2_df9b21aa-8185-40ce-8f17-dd23aed0e089/payload-processing/0.log" Sep 09 21:35:42.879216 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:42.879187 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64f7976768-h7vtq_05b70482-3a69-427f-8ce3-922c764c470b/router/0.log" Sep 09 21:35:43.550224 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:43.550194 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ncrcz_c8e97526-a820-45e2-9696-618ff30b6688/serve-healthcheck-canary/0.log" Sep 09 21:35:44.053800 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:44.053767 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-7vkst_bb25b5c9-2b1a-4286-9668-a57b10e163d6/insights-operator/0.log" Sep 09 21:35:44.055207 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:44.055188 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-7vkst_bb25b5c9-2b1a-4286-9668-a57b10e163d6/insights-operator/1.log" Sep 09 21:35:44.084027 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:44.083999 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4ckvp_9e908db9-3103-49d9-a462-b3bc3757fd89/kube-rbac-proxy/0.log" Sep 09 21:35:44.106073 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:44.106046 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4ckvp_9e908db9-3103-49d9-a462-b3bc3757fd89/exporter/0.log" Sep 09 21:35:44.128301 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:44.128276 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4ckvp_9e908db9-3103-49d9-a462-b3bc3757fd89/extractor/0.log" Sep 09 21:35:46.359955 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:46.359926 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lqck6_43cda0ce-a5ed-411b-b506-76e212d3bafc/manager/0.log" Sep 09 21:35:46.397744 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:46.397716 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vj7dn_4537c0f1-9d46-4c75-bcb2-1996f5497127/manager/0.log" Sep 09 21:35:46.436151 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:46.436110 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-88ddf69cd-w4qmv_badb67c5-7566-4353-8736-b79a3732266f/manager/0.log" Sep 09 21:35:46.601813 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:46.601770 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66dbc47bfc-nqxgt_a66d6957-d93d-4562-bb3a-e2504df76a07/manager/0.log" Sep 09 21:35:46.639130 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:46.639032 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-66dbc47bfc-nqxgt_a66d6957-d93d-4562-bb3a-e2504df76a07/copy-manifests/0.log" Sep 09 21:35:47.170277 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:47.170251 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-znttn/perf-node-gather-daemonset-dhdst" Sep 09 21:35:48.553394 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:48.553364 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-696d6796dd-thz62_51a8f793-9fae-4a02-bfc3-c5e42c7aab94/kube-auth-proxy/0.log" Sep 09 21:35:48.592243 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:48.592211 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hplk6_3a4850ff-3bce-4850-abd4-17ae634f1fa3/istio-proxy/1.log" Sep 09 21:35:48.592515 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:48.592500 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hplk6_3a4850ff-3bce-4850-abd4-17ae634f1fa3/istio-proxy/0.log" Sep 09 21:35:48.608117 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:48.608087 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-kl8v4_034864b6-654b-458e-b3de-642b0302aa3d/payload-pre-processing/0.log" Sep 09 21:35:48.621691 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:48.621669 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hplg2_df9b21aa-8185-40ce-8f17-dd23aed0e089/payload-processing/0.log" Sep 09 21:35:48.644203 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:48.644178 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64f7976768-h7vtq_05b70482-3a69-427f-8ce3-922c764c470b/router/0.log" Sep 09 21:35:49.771108 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:49.771063 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-696d6796dd-thz62_51a8f793-9fae-4a02-bfc3-c5e42c7aab94/kube-auth-proxy/0.log" Sep 09 21:35:49.813505 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:49.813474 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hplk6_3a4850ff-3bce-4850-abd4-17ae634f1fa3/istio-proxy/1.log" Sep 09 21:35:49.813887 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:49.813872 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hplk6_3a4850ff-3bce-4850-abd4-17ae634f1fa3/istio-proxy/0.log" Sep 09 21:35:49.827558 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:49.827531 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-kl8v4_034864b6-654b-458e-b3de-642b0302aa3d/payload-pre-processing/0.log" Sep 09 21:35:49.845266 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:49.845229 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-hplg2_df9b21aa-8185-40ce-8f17-dd23aed0e089/payload-processing/0.log" Sep 09 21:35:49.875563 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:49.875472 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-64f7976768-h7vtq_05b70482-3a69-427f-8ce3-922c764c470b/router/0.log" Sep 09 21:35:53.848675 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:53.848644 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-gstjf_0896f595-88a0-4e6e-956a-6abd1cbc0f97/kube-storage-version-migrator-operator/1.log" Sep 09 21:35:53.849207 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:53.849189 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-gstjf_0896f595-88a0-4e6e-956a-6abd1cbc0f97/kube-storage-version-migrator-operator/0.log" Sep 09 21:35:55.054608 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:55.054578 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxm74_8b72de97-ed02-4e23-84db-3d6a80a3e685/kube-multus-additional-cni-plugins/0.log" Sep 09 21:35:55.079956 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:55.079929 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxm74_8b72de97-ed02-4e23-84db-3d6a80a3e685/egress-router-binary-copy/0.log" Sep 09 21:35:55.101620 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:55.101592 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxm74_8b72de97-ed02-4e23-84db-3d6a80a3e685/cni-plugins/0.log" Sep 09 21:35:55.125284 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:55.125258 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxm74_8b72de97-ed02-4e23-84db-3d6a80a3e685/bond-cni-plugin/0.log" Sep 09 21:35:55.148471 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:55.148434 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxm74_8b72de97-ed02-4e23-84db-3d6a80a3e685/routeoverride-cni/0.log" Sep 09 21:35:55.171398 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:55.171372 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxm74_8b72de97-ed02-4e23-84db-3d6a80a3e685/whereabouts-cni-bincopy/0.log" Sep 09 21:35:55.193065 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:55.193032 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nxm74_8b72de97-ed02-4e23-84db-3d6a80a3e685/whereabouts-cni/0.log" Sep 09 21:35:55.424908 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:55.424833 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-j4r27_e0d6e1e2-96f1-4c46-b850-43586ded2151/kube-multus/0.log" Sep 09 21:35:55.446887 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:55.446862 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-55b8r_1abf2101-20e2-4e99-bb20-206110f86504/network-metrics-daemon/0.log" Sep 09 21:35:55.465703 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:55.465679 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-55b8r_1abf2101-20e2-4e99-bb20-206110f86504/kube-rbac-proxy/0.log" Sep 09 21:35:56.384548 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:56.384515 2553 scope.go:117] "RemoveContainer" containerID="6e846c577f1c757d52409cc592bd7022872095915ce3cfafb7df4a9c6f07735f" Sep 09 21:35:56.384961 ip-10-0-137-115 kubenswrapper[2553]: E0909 21:35:56.384689 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"main\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=main pod=e2e-distinct-simulated-kserve-6685d86975-zgwnf_llm(02188189-a3cc-4ef7-812a-5edb6ec3e3c2)\"" pod="llm/e2e-distinct-simulated-kserve-6685d86975-zgwnf" podUID="02188189-a3cc-4ef7-812a-5edb6ec3e3c2" Sep 09 21:35:56.874725 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:56.874695 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zmmsz_d70094ac-92b5-4cfd-bad5-522da65bbf9e/ovn-controller/0.log" Sep 09 21:35:56.893187 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:56.893156 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zmmsz_d70094ac-92b5-4cfd-bad5-522da65bbf9e/ovn-acl-logging/0.log" Sep 09 21:35:56.917183 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:56.917157 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zmmsz_d70094ac-92b5-4cfd-bad5-522da65bbf9e/kube-rbac-proxy-node/0.log" Sep 09 21:35:56.938790 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:56.938753 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zmmsz_d70094ac-92b5-4cfd-bad5-522da65bbf9e/kube-rbac-proxy-ovn-metrics/0.log" Sep 09 21:35:56.958934 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:56.958905 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zmmsz_d70094ac-92b5-4cfd-bad5-522da65bbf9e/northd/0.log" Sep 09 21:35:56.979809 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:56.979786 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zmmsz_d70094ac-92b5-4cfd-bad5-522da65bbf9e/nbdb/0.log" Sep 09 21:35:56.999741 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:56.999715 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zmmsz_d70094ac-92b5-4cfd-bad5-522da65bbf9e/sbdb/0.log" Sep 09 21:35:57.041600 ip-10-0-137-115 kubenswrapper[2553]: I0909 21:35:57.041570 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zmmsz_d70094ac-92b5-4cfd-bad5-522da65bbf9e/ovnkube-controller/0.log"